esp32-hacking/ROADMAP.md

# ESP32 Hacking Roadmap

## v0.1 - Documentation & Setup [DONE]
- [x] Copy firmware sources from esp-csi
- [x] Document current firmware behavior
- [x] Document CSI data format
- [x] Document build & flash process
- [x] List firmware modification ideas with priorities
- [x] Verify build from repo (ESP-IDF v5.5.2, aarch64)

## v0.2 - Remote Management [DONE]
- [x] Add UDP command listener on ESP32 (port 5501)
- [x] Implement REBOOT command
- [x] Implement IDENTIFY command (LED solid 5s)
- [x] Implement STATUS command (uptime, heap, RSSI, tx_power, rate, version)
- [x] Implement RATE command (change ping Hz, NVS persist)
- [x] Implement POWER command (TX power dBm, NVS persist)
- [x] Add LED status indicator (off/slow blink/fast blink/solid)
- [x] NVS persistence for rate and tx_power settings
- [x] Pi-side `esp-cmd` and `esp-fleet` CLI tools
- [x] mDNS hostname, watchdog, human-readable uptime
- [x] Build and flash to device
- [x] Update CHEATSHEET.md with new commands

## v0.3 - OTA Updates [DONE]
- [x] Dual OTA partition table (ota_0 + ota_1, 1920 KB each)
- [x] 4MB flash config, custom partitions in sdkconfig.defaults
- [x] OTA command handler + ota_task in firmware
- [x] LED_OTA double-blink pattern during download
- [x] Bootloader rollback on failed update (30s watchdog)
- [x] Version field in STATUS reply
- [x] Pi-side `esp-ota` tool (HTTP server + OTA orchestration)
- [x] `esp-fleet ota` subcommand (sequential fleet update)
- [x] USB-flash first device (partition table change)
- [x] End-to-end OTA test

## v0.4 - Adaptive Sampling [DONE]
- [x] On-device CSI wander calculation (coefficient of variation)
- [x] Reduce to 10 pkt/s when idle (3s holdoff)
- [x] Increase to 100 pkt/s on motion detection
- [x] Rate change EVENT notification to Pi via UDP
- [x] ADAPTIVE ON/OFF command (NVS persisted)
- [x] THRESHOLD command for tuning sensitivity (NVS persisted)
- [x] RATE command disables adaptive mode
- [x] adaptive/motion fields in STATUS reply

## v0.5 - BLE Scanning [DONE]
- [x] Enable Bluetooth alongside WiFi (NimBLE, BLE ON/OFF command)
- [x] Periodic BLE advertisement scanning
- [x] Report device MAC, RSSI, name via UDP (`BLE_DATA,<mac>,<rssi>,<type>,<name>`)
- [x] Pi-side BLE device tracking (`esp-ctl ble --track`, dedup, summary table)

## v1.0 - Production Firmware [DONE]
- [x] mDNS auto-discovery
- [x] Watchdog + auto-recovery
- [x] Configuration via UDP (`TARGET <ip> [port]`, NVS persisted)
- [x] Comprehensive error handling (watchdog, WiFi reconnect via example_connect)
- [x] `esp-ctl` unified CLI tool (cmd, status, listen, serial, ble, target, profile)
- [x] PROFILE command (heap, stack watermarks, CPU runtime stats)

## v1.1 - Passive Sensing & Multi-Sensor [DONE]
- [x] Sensor ID in data packets (hostname prefix on CSI_DATA, BLE_DATA, EVENT)
- [x] WiFi probe request capture (`PROBE_DATA,<sensor>,<mac>,<rssi>,<ssid>`, 10s dedup)
- [x] Deauth/disassoc frame detection (`ALERT_DATA,<sensor>,<type>,<sender>,<target>,<rssi>`)
- [x] BLE alerting (`esp-ctl ble --known`, alert on unknown MACs)
- [x] BLE dwell time tracking (`esp-ctl ble --track` dwell column)
- [x] Timestamped event logging (`esp-ctl listen --timestamp`)
- [x] Runtime HOSTNAME command (NVS persisted, mDNS updated)
- [x] mDNS service advertisement (`_esp-csi._udp`) + discovery (`esp-ctl discover`)
- [x] SCANRATE / PROBERATE tuning commands (NVS persisted)
- [x] Chip temperature reporting in STATUS (`temp=` field)
- [x] Channel, boot count, RSSI min/max, actual CSI rate in STATUS
- [x] FreeRTOS CPU runtime stats in PROFILE

Note: Promiscuous mode (probe/deauth capture) disabled on original ESP32 — breaks CSI callback. Works on ESP32-C6+.

## v1.2 - On-Device CSI Processing [DONE]
- [x] On-device feature extraction (amp_rms, amp_std, amp_max, amp_max_idx, energy)
- [x] CSIMODE command: RAW, COMPACT, HYBRID N (NVS persisted)
- [x] Compact payload format (`F:rms,std,max,idx,energy` — ~80% bandwidth reduction)
- [x] Hybrid mode: compact every packet, raw every Nth
- [x] Adaptive sampling reuses extracted energy (no duplicate computation)

## v1.3 - Security & OSINT [DONE]
- [x] HMAC-SHA256 command authentication (firmware + esp-ctl/esp-cmd/esp-fleet/esp-ota)
- [x] AUTH command (set/query/disable secret, NVS persisted)
- [x] Deauth flood detection (ring buffer, aggregate ALERT_DATA)
- [x] FLOODTHRESH command (count + window, NVS persisted)
- [x] MAC OUI vendor lookup (`esp-ctl oui`, IEEE CSV database)
- [x] OSINT SQLite database (probe_ssids, device_sightings tables)
- [x] Watch daemon (`esp-ctl watch` — listen + enrich + store)
- [x] OSINT query CLI (`esp-ctl osint probes/devices/mac/stats`)
- [x] Home Assistant webhook integration (deauth_flood, unknown_probe, unknown_ble)
- [x] Parallel OTA fleet updates (`esp-fleet ota --parallel`)

## v1.4 - Multi-Sensor & Validation [DONE]
- [x] Multi-sensor BLE correlation in esp-ctl (zone tracking by source sensor)
- [x] Zone tracking with EMA RSSI (`esp-ctl osint zones`, `device_zones` table)
- [x] Per-sensor breakdown in MAC profile (`esp-ctl osint mac`)
- [x] POWERTEST command (7-phase power profiling with EVENT markers)

## v1.5 - Event Handling & NVS Persistence [DONE]
- [x] EVENT packet parsing in watch daemon (motion, wifi_reconnect, powertest)
- [x] Sensor heartbeat tracking (offline/online detection, configurable timeout)
- [x] sensor_events table with indexed queries (`esp-ctl osint events`)
- [x] HA webhooks for sensor_offline, sensor_online, motion_change
- [x] NVS persistence for SCANRATE and PROBERATE commands
- [x] Sensor events count in `esp-ctl osint stats`

## v1.6 - Power Management [DONE]
- [x] ESP-IDF power management framework (DFS 240/80 MHz + light sleep)
- [x] POWERSAVE command (WiFi modem sleep toggle, NVS persisted, default off)
- [x] POWERTEST save/restore of powersave state
- [x] sdkconfig: CONFIG_PM_ENABLE, CONFIG_FREERTOS_USE_TICKLESS_IDLE

## v1.7 - Baseline Calibration & Presence Detection [DONE]
- [x] CALIBRATE command (capture N seconds of CSI with room empty, average per-subcarrier amplitudes, store in NVS)
- [x] CALIBRATE STATUS / CALIBRATE CLEAR subcommands
- [x] Presence scoring (normalized Euclidean distance of live CSI vs baseline, rolling window)
- [x] PRESENCE ON/OFF command (NVS persisted, requires valid baseline)
- [x] PRESENCE THRESHOLD command (tunable 0.001-1.0, NVS persisted)
- [x] Presence events (`EVENT,<hostname>,presence=<0|1> score=<float>`) with 10s holdoff
- [x] Calibration done event (`EVENT,<hostname>,calibrate=done packets=<n> nsub=<n>`)
- [x] presence= and pr_score= fields in STATUS reply
- [x] NVS persistence for baseline (bl_amps blob, bl_nsub) and presence config

## v1.8 - HTTPS OTA Support [DONE]
- [x] Support HTTPS URLs for OTA updates (esp_https_ota)
- [x] CI: Upload firmware to Gitea releases for OTA
- [x] CI: Simplify deploy script to pure POSIX sh

## v1.9 - Multi-Channel Scanning & BLE Fingerprinting [DONE]
- [x] CHANSCAN command (ON/OFF/NOW/INTERVAL)
- [x] Channel hopping (1-13) with 100ms dwell, pause CSI during scan
- [x] chanscan= field in STATUS, chanscan_int NVS persistence
- [x] BLE fingerprinting: company_id, tx_power, adv_flags in BLE_DATA
- [x] Historical presence sessions support

## v1.10 - LED Quiet Mode & CI Hardening [DONE]
- [x] LED quiet mode (off normally, solid on motion/presence, blinks on OTA)
- [x] Default LED to quiet mode
- [x] Build metadata in STATUS (date, time, IDF version, chip info)
- [x] CI security checks (secrets scan, config validation, size check)
- [x] Size optimization (`-Os`, saves ~75KB vs -O2)
- [x] CSI ON/OFF toggle command (NVS persisted)

## v1.11 - Diagnostics & Usability [DONE]
- [x] HELP command (lists all commands with syntax)
- [x] CONFIG command (dump all running config key=value)
- [x] FACTORY command (erase NVS config + reboot)
- [x] PING command (echo reply for connectivity tests)
- [x] LOG command (runtime log level control)
- [x] RSSI RESET command (reset min/max counters)
- [x] OTA rollback validation (crasher firmware + bootloader rollback confirmed)
- [x] Tagged v1.11.0 and OTA deployed to all 3 sensors

## v1.12 - Monitoring & Multi-Target (unreleased)
- [x] ALERT command (temp/heap thresholds, EVENT emission, 60s holdoff, NVS persisted)
- [ ] Multi-target (send data to 2+ UDP destinations)

## Web Backend (`~/git/esp32-web/`)

Tracked in its own repository. See `~/git/esp32-web/ROADMAP.md`.

Current: v0.1.5 (zones, intelligence dashboard, fleet management, 77 tests).
Next: v0.1.6 (auth, rate limiting, production deployment).

## v3.0 - Hardware Upgrade (ESP32-S3/C6)

Requires replacing current ESP32 (original) DevKitC V1 boards with ESP32-S3
or ESP32-C6 modules. The original ESP32 lacks FTM and has CSI/promiscuous
mode conflicts.

- [ ] Select target chip (ESP32-S3 for dual-core + BLE 5, or C6 for WiFi 6 + 802.15.4)
- [ ] Port firmware to new target (`idf.py set-target`, adjust `#if CONFIG_IDF_TARGET_*`)
- [ ] WiFi FTM / 802.11mc support (Fine Timing Measurement, ~1-2m accuracy)
  - FTM initiator + responder mode on each sensor
  - Inter-sensor ranging (3 pairs from 3 sensors)
  - Auto-calibrate sensor positions for 3D floor plan
- [ ] Enable promiscuous mode alongside CSI (works on S2/S3/C3/C6)
- [ ] Validate CSI quality on new chip (subcarrier count differs)
- [ ] Update parsers for chip-specific CSI format

## Future
- AP+STA config portal (WIFI_MODE_APSTA, captive portal for initial setup)
- ESP-NOW mesh (ESP32-to-ESP32 CSI)
- External sensor support (PIR, temp/humidity via GPIO)
- Pin mapping for ESP32-DevKitC V1
- Compare CSI quality: passive (router) vs active (ESP-NOW)
- Multi-sensor deployment guide (placement, zones, triangulation)
- Home Assistant integration (MQTT discovery, entity creation)
- Grafana dashboards for long-term analytics
- ML-based device classification (phone vs laptop vs IoT)
- RSSI triangulation with 3+ sensors (approximate device location)