feat: Add v0.3 OTA updates — dual partition, esp-ota tool, rollback

Dual OTA partition table (ota_0/ota_1, 1920 KB each) on 4MB flash.
Firmware gains OTA command, LED_OTA double-blink, version in STATUS,
and automatic rollback validation. Pi-side esp-ota tool serves firmware
via HTTP and orchestrates the update flow. esp-fleet gains ota subcommand.
This commit is contained in:
user
2026-02-04 16:19:09 +01:00
parent 44bd549761
commit d65ac208b9
8 changed files with 368 additions and 36 deletions

View File

@@ -8,25 +8,31 @@
- [x] List firmware modification ideas with priorities
- [x] Verify build from repo (ESP-IDF v5.5.2, aarch64)
## v0.2 - Remote Management [IN PROGRESS]
## v0.2 - Remote Management [DONE]
- [x] Add UDP command listener on ESP32 (port 5501)
- [x] Implement REBOOT command
- [x] Implement IDENTIFY command (LED solid 5s)
- [x] Implement STATUS command (uptime, heap, RSSI, tx_power, rate)
- [x] Implement STATUS command (uptime, heap, RSSI, tx_power, rate, version)
- [x] Implement RATE command (change ping Hz, NVS persist)
- [x] Implement POWER command (TX power dBm, NVS persist)
- [x] Add LED status indicator (off/slow blink/fast blink/solid)
- [x] NVS persistence for rate and tx_power settings
- [x] Pi-side `esp-cmd` CLI tool
- [ ] Build and flash test on device
- [ ] Update CHEATSHEET.md with new commands
- [x] Pi-side `esp-cmd` and `esp-fleet` CLI tools
- [x] mDNS hostname, watchdog, human-readable uptime
- [x] Build and flash to device
- [x] Update CHEATSHEET.md with new commands
## v0.3 - OTA Updates
- [ ] Modify partition table for dual OTA
- [ ] Add OTA update handler in firmware
- [ ] HTTP firmware server on Pi
- [ ] Pi-side flash script (select device, push firmware)
- [ ] Rollback on failed update
## v0.3 - OTA Updates [IN PROGRESS]
- [x] Dual OTA partition table (ota_0 + ota_1, 1920 KB each)
- [x] 4MB flash config, custom partitions in sdkconfig.defaults
- [x] OTA command handler + ota_task in firmware
- [x] LED_OTA double-blink pattern during download
- [x] Bootloader rollback on failed update (30s watchdog)
- [x] Version field in STATUS reply
- [x] Pi-side `esp-ota` tool (HTTP server + OTA orchestration)
- [x] `esp-fleet ota` subcommand (sequential fleet update)
- [ ] USB-flash first device (partition table change)
- [ ] End-to-end OTA test
## v0.4 - Adaptive Sampling
- [ ] On-device wander calculation (simplified)
@@ -41,8 +47,8 @@
- [ ] Pi-side BLE device tracking
## v1.0 - Production Firmware
- [ ] mDNS auto-discovery
- [ ] Watchdog + auto-recovery
- [x] mDNS auto-discovery (done in v0.2)
- [x] Watchdog + auto-recovery (done in v0.2)
- [ ] On-device CSI processing (send metrics, not raw)
- [ ] Configuration via UDP (WiFi SSID, target IP)
- [ ] Comprehensive error handling

View File

@@ -2,30 +2,45 @@
**Last Updated:** 2026-02-04
## Current Sprint: v0.2 - Remote Management
## Current Sprint: v0.3 - OTA Updates
### P0 - Critical
- [x] Firmware: UDP command listener (port 5501)
- [x] Firmware: LED status indicator (GPIO2)
- [x] Firmware: NVS config persistence (rate, tx_power)
- [~] Build and flash firmware to device
- [x] Create dual OTA partition table (`partitions.csv`)
- [x] Update `sdkconfig.defaults` (4MB flash, custom partitions, rollback, HTTP OTA)
- [x] Firmware: OTA command, ota_task, LED_OTA, rollback validation
- [x] Firmware: Add version to STATUS reply
- [ ] `idf.py reconfigure` to regenerate sdkconfig
- [ ] Build and USB-flash first device (partition table change requires USB)
### P1 - Important
- [x] Firmware: REBOOT, IDENTIFY, STATUS commands
- [x] Firmware: RATE command (10-100 Hz, restarts ping)
- [x] Firmware: POWER command (2-20 dBm)
- [x] Firmware: Refactor ping to support restart
- [x] Pi-side: `esp-cmd` CLI tool
- [ ] Update CHEATSHEET.md with esp-cmd usage
- [x] Pi-side `esp-ota` tool (HTTP server + OTA orchestration)
- [x] `esp-fleet ota` subcommand (sequential fleet OTA)
- [ ] Test OTA end-to-end: `esp-ota amber-maple.local`
- [ ] Regenerate `sdkconfig.sample`
### P2 - Normal
- [ ] Document esp-cmd in USAGE.md
- [ ] Add Kconfig CSI_CMD_PORT option
- [ ] OTA update remaining fleet (muddy-storm, hollow-acorn) via USB
- [ ] Test rollback (flash bad firmware, verify auto-revert)
- [ ] Document esp-ota in USAGE.md
### P3 - Low
- [ ] Document esp-crab dual-antenna capabilities
- [ ] Document esp-radar console features
## Completed: v0.2 - Remote Management
- [x] Firmware: UDP command listener (port 5501)
- [x] Firmware: LED status indicator (GPIO2)
- [x] Firmware: NVS config persistence (rate, tx_power)
- [x] Firmware: REBOOT, IDENTIFY, STATUS commands
- [x] Firmware: RATE command (10-100 Hz, restarts ping)
- [x] Firmware: POWER command (2-20 dBm)
- [x] Pi-side: `esp-cmd` CLI tool
- [x] Pi-side: `esp-fleet` fleet management tool
- [x] mDNS hostname, watchdog, human-readable uptime
- [x] Build and flash to device
- [x] Update CHEATSHEET.md
## Completed: v0.1 - Documentation
- [x] Copy firmware sources to project
@@ -34,12 +49,11 @@
- [x] Create .gitignore for build artifacts
- [x] Test building firmware from this repo
- [x] Document CSI config options
- [x] Compare csi_recv vs csi_recv_router differences
## Notes
- Build confirmed working on ESP-IDF v5.5.2 (aarch64/Pi 5)
- v0.2 firmware adds ~1.5 KB heap + 6 KB stack usage
- NVS namespace: `csi_config` (keys: `send_rate`, `tx_power`)
- LED uses GPIO2 (built-in on most ESP32 dev boards)
- Command port default: 5501 (configurable via menuconfig)
- NVS offset changes with new partition table — first USB flash resets saved config
- First device must be USB-flashed to switch partition table, subsequent updates via OTA
- `esp_https_ota` is built into ESP-IDF core — no extra deps needed
- OTA download ~790 KB on LAN takes ~3-5s, well under 30s watchdog
- CSI data keeps flowing during OTA download

View File

@@ -34,10 +34,11 @@ idf.py reconfigure # Re-fetch managed components
## Remote Management (esp-cmd)
```bash
esp-cmd <host> STATUS # Uptime, heap, RSSI, tx_power, rate
esp-cmd <host> STATUS # Uptime, heap, RSSI, tx_power, rate, version
esp-cmd <host> IDENTIFY # LED solid 5s (find the device)
esp-cmd <host> RATE 50 # Set ping rate to 50 Hz (NVS saved)
esp-cmd <host> POWER 15 # Set TX power to 15 dBm (NVS saved)
esp-cmd <host> OTA http://pi:8070/fw # Trigger OTA update (use esp-ota instead)
esp-cmd <host> REBOOT # Restart device
```
@@ -55,8 +56,34 @@ esp-fleet status # Query all sensors at once
esp-fleet identify # Blink all LEDs
esp-fleet rate 50 # Set rate on all devices
esp-fleet reboot # Reboot entire fleet
esp-fleet ota # OTA update all (sequential)
esp-fleet ota /path/to/firmware.bin # OTA with custom firmware
```
## OTA Updates (esp-ota)
```bash
esp-ota amber-maple.local # OTA with default build
esp-ota amber-maple.local -f fw.bin # OTA with custom firmware
esp-ota amber-maple.local --no-wait # Fire and forget
```
**First flash after enabling OTA requires USB** (partition table change).
After that, all updates are OTA.
### OTA Flow
1. Verifies device is alive via STATUS
2. Starts temp HTTP server on Pi (port 8070)
3. Sends `OTA http://<pi>:8070/<fw>.bin` via UDP
4. Device downloads, flashes, reboots
5. Verifies device responds post-reboot
### Rollback
If new firmware crashes or hangs, the 30s watchdog reboots and bootloader
automatically rolls back to the previous firmware.
### LED States
| LED | Meaning |
@@ -65,6 +92,7 @@ esp-fleet reboot # Reboot entire fleet
| Slow blink (1 Hz) | Connected, no CSI activity |
| Fast blink (5 Hz) | CSI data flowing |
| Solid (5s) | IDENTIFY command active |
| Double blink | OTA in progress |
## Test CSI Reception
@@ -109,6 +137,7 @@ secondary_channel,timestamp,ant,sig_len,rx_state,len,first_word,"[I,Q,...]"
| `get-started/csi_recv_router/main/app_main.c` | Main firmware source |
| `get-started/csi_recv_router/main/Kconfig.projbuild` | UDP/cmd port config |
| `tools/esp-cmd` | Pi-side management CLI |
| `tools/esp-ota` | Pi-side OTA update tool |
| `tools/esp-fleet` | Fleet-wide command tool |
| `get-started/csi_recv_router/sdkconfig.defaults` | SDK defaults |
| `get-started/csi_recv_router/main/idf_component.yml` | Dependencies |

View File

@@ -30,6 +30,9 @@
#include "esp_now.h"
#include "esp_timer.h"
#include "esp_task_wdt.h"
#include "esp_ota_ops.h"
#include "esp_https_ota.h"
#include "esp_http_client.h"
#include "driver/gpio.h"
#include "mdns.h"
@@ -65,6 +68,7 @@ typedef enum {
LED_SLOW_BLINK,
LED_FAST_BLINK,
LED_SOLID,
LED_OTA,
} led_mode_t;
/* --- Globals --- */
@@ -74,6 +78,7 @@ static esp_ping_handle_t s_ping_handle = NULL;
static volatile led_mode_t s_led_mode = LED_OFF;
static volatile int64_t s_last_csi_time = 0;
static volatile int64_t s_identify_end_time = 0;
static volatile bool s_ota_in_progress = false;
/* UDP socket for CSI data transmission */
static int s_udp_socket = -1;
@@ -181,6 +186,18 @@ static void led_task(void *arg)
led_on = true;
vTaskDelay(pdMS_TO_TICKS(200));
break;
case LED_OTA:
/* Double-blink: on-off-on-off-pause */
gpio_set_level(LED_GPIO, 1);
vTaskDelay(pdMS_TO_TICKS(80));
gpio_set_level(LED_GPIO, 0);
vTaskDelay(pdMS_TO_TICKS(80));
gpio_set_level(LED_GPIO, 1);
vTaskDelay(pdMS_TO_TICKS(80));
gpio_set_level(LED_GPIO, 0);
vTaskDelay(pdMS_TO_TICKS(500));
led_on = false;
break;
}
}
}
@@ -375,6 +392,41 @@ static esp_err_t wifi_ping_router_start(void)
return ESP_OK;
}
/* --- OTA --- */
static void ota_task(void *arg)
{
char *url = (char *)arg;
ESP_LOGI(TAG, "OTA: downloading from %s", url);
s_led_mode = LED_OTA;
esp_http_client_config_t http_cfg = {
.url = url,
.timeout_ms = 30000,
};
esp_https_ota_config_t ota_cfg = {
.http_config = &http_cfg,
};
esp_err_t err = esp_https_ota(&ota_cfg);
free(url);
if (err == ESP_OK) {
ESP_LOGI(TAG, "OTA: success, rebooting...");
s_led_mode = LED_SOLID;
vTaskDelay(pdMS_TO_TICKS(500));
esp_restart();
} else {
ESP_LOGE(TAG, "OTA: failed: %s", esp_err_to_name(err));
s_led_mode = LED_SLOW_BLINK;
s_ota_in_progress = false;
}
vTaskDelete(NULL);
}
/* --- Command handler --- */
static void reboot_after_delay(void *arg)
@@ -414,6 +466,8 @@ static int cmd_handle(const char *cmd, char *reply, size_t reply_size)
rssi = ap.rssi;
}
const esp_app_desc_t *app_desc = esp_app_get_description();
char uptime_str[32];
if (days > 0) {
snprintf(uptime_str, sizeof(uptime_str), "%dd%dh%dm", days, hours, mins);
@@ -424,9 +478,9 @@ static int cmd_handle(const char *cmd, char *reply, size_t reply_size)
}
snprintf(reply, reply_size,
"OK STATUS uptime=%s heap=%lu rssi=%d tx_power=%d rate=%d hostname=%s",
"OK STATUS uptime=%s heap=%lu rssi=%d tx_power=%d rate=%d hostname=%s version=%s",
uptime_str, (unsigned long)heap, rssi, (int)s_tx_power_dbm,
s_send_frequency, CONFIG_CSI_HOSTNAME);
s_send_frequency, CONFIG_CSI_HOSTNAME, app_desc->version);
return strlen(reply);
}
@@ -458,6 +512,28 @@ static int cmd_handle(const char *cmd, char *reply, size_t reply_size)
return strlen(reply);
}
/* OTA <url> */
if (strncmp(cmd, "OTA ", 4) == 0) {
const char *url = cmd + 4;
if (strncmp(url, "http://", 7) != 0) {
snprintf(reply, reply_size, "ERR OTA url must start with http://");
return strlen(reply);
}
if (s_ota_in_progress) {
snprintf(reply, reply_size, "ERR OTA already in progress");
return strlen(reply);
}
char *url_copy = strdup(url);
if (!url_copy) {
snprintf(reply, reply_size, "ERR OTA out of memory");
return strlen(reply);
}
s_ota_in_progress = true;
xTaskCreate(ota_task, "ota_task", 8192, url_copy, 5, NULL);
snprintf(reply, reply_size, "OK OTA started");
return strlen(reply);
}
snprintf(reply, reply_size, "ERR UNKNOWN");
return strlen(reply);
}
@@ -563,4 +639,14 @@ void app_main()
wifi_ping_router_start();
xTaskCreate(cmd_task, "cmd_task", 4096, NULL, 5, NULL);
/* OTA rollback: mark firmware valid if we got this far */
const esp_partition_t *running = esp_ota_get_running_partition();
esp_ota_img_states_t ota_state;
if (esp_ota_get_state_partition(running, &ota_state) == ESP_OK) {
if (ota_state == ESP_OTA_IMG_PENDING_VERIFY) {
ESP_LOGI(TAG, "OTA: marking firmware valid (rollback cancelled)");
esp_ota_mark_app_valid_cancel_rollback();
}
}
}

View File

@@ -0,0 +1,6 @@
# Name, Type, SubType, Offset, Size
nvs, data, nvs, 0x9000, 0x4000
otadata, data, ota, 0xd000, 0x2000
phy_init, data, phy, 0xf000, 0x1000
ota_0, app, ota_0, 0x10000, 0x1E0000
ota_1, app, ota_1, 0x1F0000, 0x1E0000
1 # Name Type SubType Offset Size
2 nvs data nvs 0x9000 0x4000
3 otadata data ota 0xd000 0x2000
4 phy_init data phy 0xf000 0x1000
5 ota_0 app ota_0 0x10000 0x1E0000
6 ota_1 app ota_1 0x1F0000 0x1E0000

View File

@@ -49,3 +49,16 @@ CONFIG_ESP32_WIFI_DYNAMIC_TX_BUFFER_NUM=32
#
CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240=y
CONFIG_ESP32S3_DEFAULT_CPU_FREQ_MHZ=240
#
# Flash & Partitions (4MB flash, dual OTA)
#
CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y
CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv"
#
# OTA Updates
#
CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE=y
CONFIG_ESP_HTTPS_OTA_ALLOW_HTTP=y

View File

@@ -2,7 +2,9 @@
"""Query all ESP32 CSI sensors in parallel."""
import concurrent.futures
import os
import socket
import subprocess
import sys
DEFAULT_PORT = 5501
@@ -14,6 +16,8 @@ SENSORS = [
("hollow-acorn", "hollow-acorn.local"),
]
ESP_OTA = os.path.join(os.path.dirname(os.path.abspath(__file__)), "esp-ota")
USAGE = """\
Usage: esp-fleet <command> [args...]
@@ -25,11 +29,14 @@ Commands:
rate <10-100> Set ping rate on all devices
power <2-20> Set TX power on all devices
reboot Reboot all devices
ota [firmware.bin] OTA update all devices (sequentially)
Examples:
esp-fleet status
esp-fleet identify
esp-fleet rate 50"""
esp-fleet rate 50
esp-fleet ota
esp-fleet ota /path/to/firmware.bin"""
def query(name, host, cmd):
@@ -54,11 +61,33 @@ def query(name, host, cmd):
sock.close()
def run_ota(firmware=None):
"""Run OTA on each sensor sequentially."""
for name, host in SENSORS:
print(f"\n{'='*40}")
print(f"OTA: {name} ({host})")
print(f"{'='*40}")
cmd = [ESP_OTA, host]
if firmware:
cmd += ["-f", firmware]
result = subprocess.run(cmd)
if result.returncode != 0:
print(f"ERR: OTA failed for {name}, stopping fleet OTA", file=sys.stderr)
sys.exit(1)
print(f"\nAll {len(SENSORS)} devices updated.")
def main():
if len(sys.argv) < 2 or sys.argv[1] in ("-h", "--help"):
print(USAGE)
sys.exit(0 if sys.argv[1:] and sys.argv[1] in ("-h", "--help") else 2)
# Handle OTA subcommand separately (sequential, not parallel)
if sys.argv[1].lower() == "ota":
firmware = sys.argv[2] if len(sys.argv) > 2 else None
run_ota(firmware)
return
cmd = " ".join(sys.argv[1:]).strip().upper()
with concurrent.futures.ThreadPoolExecutor(max_workers=len(SENSORS)) as pool:

149
tools/esp-ota Executable file
View File

@@ -0,0 +1,149 @@
#!/usr/bin/env python3
"""OTA firmware update for ESP32 CSI devices."""
import argparse
import http.server
import os
import socket
import sys
import threading
import time
DEFAULT_CMD_PORT = 5501
DEFAULT_HTTP_PORT = 8070
DEFAULT_FW = os.path.expanduser(
"~/git/esp32-hacking/get-started/csi_recv_router/build/csi_recv_router.bin"
)
TIMEOUT = 2.0
REBOOT_WAIT = 25.0
STATUS_RETRIES = 10
STATUS_INTERVAL = 3.0
def resolve(host: str) -> str:
"""Resolve hostname to IP address."""
try:
result = socket.getaddrinfo(host, DEFAULT_CMD_PORT, socket.AF_INET, socket.SOCK_DGRAM)
return result[0][4][0]
except socket.gaierror as e:
print(f"ERR: cannot resolve {host}: {e}", file=sys.stderr)
sys.exit(1)
def udp_cmd(ip: str, cmd: str, timeout: float = TIMEOUT) -> str:
"""Send UDP command and return reply."""
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.settimeout(timeout)
try:
sock.sendto(cmd.encode(), (ip, DEFAULT_CMD_PORT))
data, _ = sock.recvfrom(512)
return data.decode().strip()
finally:
sock.close()
def get_local_ip(target_ip: str) -> str:
"""Determine which local IP the target can reach us on."""
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
sock.connect((target_ip, 80))
return sock.getsockname()[0]
finally:
sock.close()
def serve_firmware(directory: str, port: int) -> http.server.HTTPServer:
"""Start HTTP server serving firmware directory in a background thread."""
handler = lambda *a, **k: http.server.SimpleHTTPRequestHandler(
*a, directory=directory, **k
)
server = http.server.HTTPServer(("0.0.0.0", port), handler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
return server
def main():
parser = argparse.ArgumentParser(description="OTA firmware update for ESP32 CSI devices")
parser.add_argument("host", help="Device hostname or IP (e.g., amber-maple.local)")
parser.add_argument("-f", "--firmware", default=DEFAULT_FW, help="Path to firmware .bin")
parser.add_argument("-p", "--port", type=int, default=DEFAULT_HTTP_PORT, help="HTTP server port")
parser.add_argument("--no-wait", action="store_true", help="Don't wait for reboot verification")
args = parser.parse_args()
fw_path = os.path.abspath(args.firmware)
if not os.path.isfile(fw_path):
print(f"ERR: firmware not found: {fw_path}", file=sys.stderr)
sys.exit(1)
fw_size = os.path.getsize(fw_path)
fw_dir = os.path.dirname(fw_path)
fw_name = os.path.basename(fw_path)
print(f"Firmware: {fw_path} ({fw_size // 1024} KB)")
# Resolve device
ip = resolve(args.host)
print(f"Device: {args.host} ({ip})")
# Check device is alive
try:
reply = udp_cmd(ip, "STATUS")
print(f"Status: {reply}")
except (socket.timeout, OSError) as e:
print(f"ERR: device not responding: {e}", file=sys.stderr)
sys.exit(1)
# Determine local IP
local_ip = get_local_ip(ip)
ota_url = f"http://{local_ip}:{args.port}/{fw_name}"
print(f"OTA URL: {ota_url}")
# Start HTTP server
server = serve_firmware(fw_dir, args.port)
print(f"HTTP server on port {args.port}")
# Send OTA command
try:
reply = udp_cmd(ip, f"OTA {ota_url}")
print(f"OTA cmd: {reply}")
except (socket.timeout, OSError) as e:
server.shutdown()
print(f"ERR: OTA command failed: {e}", file=sys.stderr)
sys.exit(1)
if not reply.startswith("OK"):
server.shutdown()
print(f"ERR: device rejected OTA: {reply}", file=sys.stderr)
sys.exit(1)
if args.no_wait:
print("OTA started (--no-wait, not verifying)")
# Keep server alive briefly for download
time.sleep(30)
server.shutdown()
return
# Wait for device to download, flash, and reboot
print(f"Waiting for reboot...")
time.sleep(REBOOT_WAIT)
# Verify device comes back
for attempt in range(1, STATUS_RETRIES + 1):
try:
ip = resolve(args.host)
reply = udp_cmd(ip, "STATUS", timeout=3.0)
print(f"Verified: {reply}")
server.shutdown()
return
except (socket.timeout, OSError):
print(f" retry {attempt}/{STATUS_RETRIES}...")
time.sleep(STATUS_INTERVAL)
server.shutdown()
print("ERR: device did not come back after OTA", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()