esp32-hacking/ROADMAP.md

ESP32 Hacking Roadmap

v0.1 - Documentation & Setup [DONE]

• Copy firmware sources from esp-csi
• Document current firmware behavior
• Document CSI data format
• Document build & flash process
• List firmware modification ideas with priorities
• Verify build from repo (ESP-IDF v5.5.2, aarch64)

v0.2 - Remote Management [DONE]

• Add UDP command listener on ESP32 (port 5501)
• Implement REBOOT command
• Implement IDENTIFY command (LED solid 5s)
• Implement STATUS command (uptime, heap, RSSI, tx_power, rate, version)
• Implement RATE command (change ping Hz, NVS persist)
• Implement POWER command (TX power dBm, NVS persist)
• Add LED status indicator (off/slow blink/fast blink/solid)
• NVS persistence for rate and tx_power settings
• Pi-side esp-cmd and esp-fleet CLI tools
• mDNS hostname, watchdog, human-readable uptime
• Build and flash to device
• Update CHEATSHEET.md with new commands

v0.3 - OTA Updates [DONE]

• Dual OTA partition table (ota_0 + ota_1, 1920 KB each)
• 4MB flash config, custom partitions in sdkconfig.defaults
• OTA command handler + ota_task in firmware
• LED_OTA double-blink pattern during download
• Bootloader rollback on failed update (30s watchdog)
• Version field in STATUS reply
• Pi-side esp-ota tool (HTTP server + OTA orchestration)
• esp-fleet ota subcommand (sequential fleet update)
• USB-flash first device (partition table change)
• End-to-end OTA test

v0.4 - Adaptive Sampling [DONE]

• On-device CSI wander calculation (coefficient of variation)
• Reduce to 10 pkt/s when idle (3s holdoff)
• Increase to 100 pkt/s on motion detection
• Rate change EVENT notification to Pi via UDP
• ADAPTIVE ON/OFF command (NVS persisted)
• THRESHOLD command for tuning sensitivity (NVS persisted)
• RATE command disables adaptive mode
• adaptive/motion fields in STATUS reply

v0.5 - BLE Scanning [DONE]

• Enable Bluetooth alongside WiFi (NimBLE, BLE ON/OFF command)
• Periodic BLE advertisement scanning
• Report device MAC, RSSI, name via UDP (BLE_DATA,<mac>,<rssi>,<type>,<name>)
• Pi-side BLE device tracking (esp-ctl ble --track, dedup, summary table)

v1.0 - Production Firmware [DONE]

• mDNS auto-discovery
• Watchdog + auto-recovery
• Configuration via UDP (TARGET <ip> [port], NVS persisted)
• Comprehensive error handling (watchdog, WiFi reconnect via example_connect)
• esp-ctl unified CLI tool (cmd, status, listen, serial, ble, target, profile)
• PROFILE command (heap, stack watermarks, CPU runtime stats)

v1.1 - Passive Sensing & Multi-Sensor [DONE]

• Sensor ID in data packets (hostname prefix on CSI_DATA, BLE_DATA, EVENT)
• WiFi probe request capture (PROBE_DATA,<sensor>,<mac>,<rssi>,<ssid>, 10s dedup)
• Deauth/disassoc frame detection (ALERT_DATA,<sensor>,<type>,<sender>,<target>,<rssi>)
• BLE alerting (esp-ctl ble --known, alert on unknown MACs)
• BLE dwell time tracking (esp-ctl ble --track dwell column)
• Timestamped event logging (esp-ctl listen --timestamp)
• Runtime HOSTNAME command (NVS persisted, mDNS updated)
• mDNS service advertisement (_esp-csi._udp) + discovery (esp-ctl discover)
• SCANRATE / PROBERATE tuning commands (NVS persisted)
• Chip temperature reporting in STATUS (temp= field)
• Channel, boot count, RSSI min/max, actual CSI rate in STATUS
• FreeRTOS CPU runtime stats in PROFILE

Note: Promiscuous mode (probe/deauth capture) disabled on original ESP32 — breaks CSI callback. Works on ESP32-C6+.

v1.2 - On-Device CSI Processing [DONE]

• On-device feature extraction (amp_rms, amp_std, amp_max, amp_max_idx, energy)
• CSIMODE command: RAW, COMPACT, HYBRID N (NVS persisted)
• Compact payload format (F:rms,std,max,idx,energy — ~80% bandwidth reduction)
• Hybrid mode: compact every packet, raw every Nth
• Adaptive sampling reuses extracted energy (no duplicate computation)

v1.3 - Security & OSINT [DONE]

• HMAC-SHA256 command authentication (firmware + esp-ctl/esp-cmd/esp-fleet/esp-ota)
• AUTH command (set/query/disable secret, NVS persisted)
• Deauth flood detection (ring buffer, aggregate ALERT_DATA)
• FLOODTHRESH command (count + window, NVS persisted)
• MAC OUI vendor lookup (esp-ctl oui, IEEE CSV database)
• OSINT SQLite database (probe_ssids, device_sightings tables)
• Watch daemon (esp-ctl watch — listen + enrich + store)
• OSINT query CLI (esp-ctl osint probes/devices/mac/stats)
• Home Assistant webhook integration (deauth_flood, unknown_probe, unknown_ble)
• Parallel OTA fleet updates (esp-fleet ota --parallel)

v1.4 - Multi-Sensor & Validation [DONE]

• Multi-sensor BLE correlation in esp-ctl (zone tracking by source sensor)
• Zone tracking with EMA RSSI (esp-ctl osint zones, device_zones table)
• Per-sensor breakdown in MAC profile (esp-ctl osint mac)
• POWERTEST command (7-phase power profiling with EVENT markers)
• Test OTA rollback (flash bad firmware, verify auto-revert)
• Create HA webhook automations for deauth_flood / unknown_probe
• Document esp-crab dual-antenna capabilities
• Document esp-radar console features

v1.5 - Event Handling & NVS Persistence [DONE]

• EVENT packet parsing in watch daemon (motion, wifi_reconnect, powertest)
• Sensor heartbeat tracking (offline/online detection, configurable timeout)
• sensor_events table with indexed queries (esp-ctl osint events)
• HA webhooks for sensor_offline, sensor_online, motion_change
• NVS persistence for SCANRATE and PROBERATE commands
• Sensor events count in esp-ctl osint stats

v1.6 - Power Management [DONE]

• ESP-IDF power management framework (DFS 240/80 MHz + light sleep)
• POWERSAVE command (WiFi modem sleep toggle, NVS persisted, default off)
• POWERTEST save/restore of powersave state
• sdkconfig: CONFIG_PM_ENABLE, CONFIG_FREERTOS_USE_TICKLESS_IDLE
• Power consumption measurements using POWERTEST + external meter
• Deep sleep mode with wake-on-CSI-motion
• Battery-optimized duty cycling

v1.7 - Baseline Calibration & Presence Detection [DONE]

• CALIBRATE command (capture N seconds of CSI with room empty, average per-subcarrier amplitudes, store in NVS)
• CALIBRATE STATUS / CALIBRATE CLEAR subcommands
• Presence scoring (normalized Euclidean distance of live CSI vs baseline, rolling window)
• PRESENCE ON/OFF command (NVS persisted, requires valid baseline)
• PRESENCE THRESHOLD command (tunable 0.001-1.0, NVS persisted)
• Presence events (EVENT,<hostname>,presence=<0|1> score=<float>) with 10s holdoff
• Calibration done event (EVENT,<hostname>,calibrate=done packets=<n> nsub=<n>)
• presence= and pr_score= fields in STATUS reply
• NVS persistence for baseline (bl_amps blob, bl_nsub) and presence config
• Tune presence threshold per room with real-world testing

v1.8 - HTTPS OTA Support [DONE]

• Support HTTPS URLs for OTA updates (esp_https_ota)
• CI: Upload firmware to Gitea releases for OTA
• CI: Simplify deploy script to pure POSIX sh

v1.9 - Multi-Channel Scanning & BLE Fingerprinting [DONE]

• CHANSCAN command (ON/OFF/NOW/INTERVAL)
• Channel hopping (1-13) with 100ms dwell, pause CSI during scan
• chanscan= field in STATUS, chanscan_int NVS persistence
• BLE fingerprinting: company_id, tx_power, adv_flags in BLE_DATA
• Historical presence sessions support

v2.0 - Flask API Backend (Purple Team)

REST API backend for OPSEC/OSINT/Purple team operations using ESP32 sensor fleet. API-first design; frontend dashboard deferred to v2.1+.

• HTTP API: TCP 5500
• UDP Collector: UDP 5500 (sensor data)
• Sensor Commands: UDP 5501 (outbound)

Phase 1: Project Setup

• Project scaffold (~/git/esp32-web/) with Flask + SQLAlchemy + Blueprints
• Database schema: sensors, devices, sightings, alerts, events, probes
• Containerfile for podman deployment
• Makefile (build, run, dev, stop, logs)
• pytest setup with fixtures

Phase 2: UDP Collector

• Async UDP listener daemon (threading or asyncio)
• Parse all sensor streams: CSI_DATA, BLE_DATA, PROBE_DATA, ALERT_DATA, EVENT
• Store to database with timestamps
• Sensor heartbeat tracking (online/offline status)
• Run as background thread alongside Flask

Phase 3: Core API Endpoints

• GET /api/v1/sensors — list sensors with status, uptime, last_seen
• GET /api/v1/sensors/<id>/status — detailed sensor info
• POST /api/v1/sensors/<id>/command — send UDP command (proxy)
• GET /api/v1/devices — list all discovered devices (BLE + WiFi)
• GET /api/v1/devices/<mac> — device profile (sightings, zones, vendor)
• GET /api/v1/alerts — alert feed with pagination + filters
• GET /api/v1/probes — probe requests with SSID enumeration
• GET /api/v1/events — sensor events (motion, presence, calibration)

Phase 4: OSINT Features

• MAC vendor lookup (IEEE OUI database)
• BLE company_id to manufacturer mapping
• GET /api/v1/devices/<mac>/profile — enriched device intel
• GET /api/v1/stats — aggregate statistics (device counts, alert counts)
• Export endpoints: GET /api/v1/export/devices.csv, .json

Phase 5: Fleet Management API

• GET /api/v1/sensors/<id>/config — current sensor configuration
• PUT /api/v1/sensors/<id>/config — update sensor settings
• POST /api/v1/sensors/<id>/ota — trigger OTA update
• POST /api/v1/sensors/<id>/calibrate — trigger baseline calibration
• GET /api/v1/sensors/<id>/history — historical metrics

v2.1 - Web Dashboard (Future)

Frontend dashboard using htmx + Pico CSS, built on top of v2.0 API.

• Live sensor status dashboard
• Device inventory table with search/filter
• Alert timeline with severity badges
• Presence heatmap per zone
• Sensor fleet management UI

v3.0 - Hardware Upgrade (ESP32-S3/C6)

Requires replacing current ESP32 (original) DevKitC V1 boards with ESP32-S3 or ESP32-C6 modules. The original ESP32 lacks FTM and has CSI/promiscuous mode conflicts.

• Select target chip (ESP32-S3 for dual-core + BLE 5, or C6 for WiFi 6 + 802.15.4)
• Port firmware to new target (idf.py set-target, adjust #if CONFIG_IDF_TARGET_*)
• WiFi FTM / 802.11mc support (Fine Timing Measurement, ~1-2m accuracy)
  • FTM initiator + responder mode on each sensor
  • Inter-sensor ranging (3 pairs from 3 sensors)
  • Auto-calibrate sensor positions for 3D floor plan
• Enable promiscuous mode alongside CSI (works on S2/S3/C3/C6)
• Validate CSI quality on new chip (subcarrier count differs)
• Update parsers for chip-specific CSI format

Future

• AP+STA config portal (WIFI_MODE_APSTA, captive portal for initial setup)
• ESP-NOW mesh (ESP32-to-ESP32 CSI)
• External sensor support (PIR, temp/humidity via GPIO)
• Pin mapping for ESP32-DevKitC V1
• Compare CSI quality: passive (router) vs active (ESP-NOW)
• Multi-sensor deployment guide (placement, zones, triangulation)
• Home Assistant integration (MQTT discovery, entity creation)
• Grafana dashboards for long-term analytics
• ML-based device classification (phone vs laptop vs IoT)
• RSSI triangulation with 3+ sensors (approximate device location)