esp32-hacking/TASKS.md

ESP32 Hacking Tasks

Last Updated: 2026-02-05

Current Sprint: v2.0 — Flask API Backend

P0 - Critical (Phase 1: Project Setup)

• Create project scaffold ~/git/esp32-web/
• Flask app factory pattern with Blueprints
• HTTP API on TCP 5500, UDP collector on UDP 5500
• SQLAlchemy models: Sensor, Device, Sighting, Alert, Event, Probe
• Containerfile for podman
• Makefile (build, run, dev, stop, logs)
• Basic pytest setup

P1 - High (Phase 2: UDP Collector)

• UDP listener thread (parse CSI_DATA, BLE_DATA, PROBE_DATA, ALERT_DATA, EVENT)
• Store parsed data to SQLite/PostgreSQL
• Sensor heartbeat tracking (mark online/offline)
• Integrate collector with Flask app lifecycle

P1 - High (Phase 3: Core API)

• GET /api/v1/sensors — list sensors
• GET /api/v1/devices — list devices (BLE + WiFi MACs)
• GET /api/v1/alerts — alert feed with pagination
• GET /api/v1/probes — probe requests
• GET /api/v1/events — sensor events
• POST /api/v1/sensors/<id>/command — send command to sensor

P2 - Normal (Phase 4: OSINT)

• MAC vendor lookup (OUI database)
• BLE company_id mapping
• GET /api/v1/stats — aggregate statistics
• Export endpoints (CSV, JSON)

P2 - Normal (Backlog from v1.x)

• Tune presence threshold per room with real-world testing
• Power consumption measurements using POWERTEST + external meter
• Test OTA rollback (flash bad firmware, verify auto-revert)

P3 - Low

• Deep sleep mode with wake-on-CSI-motion
• Battery-optimized duty cycling
• Document esp-crab dual-antenna capabilities
• Document esp-radar console features
• Pin mapping for ESP32-DevKitC V1

Completed: v1.9 - Multi-Channel Scanning & BLE Fingerprinting

• CHANSCAN command (ON/OFF/NOW/INTERVAL)
• Channel hopping (1-13) with 100ms dwell, pause CSI during scan
• chanscan= field in STATUS, chanscan_int NVS persistence
• BLE fingerprinting: company_id, tx_power, adv_flags in BLE_DATA
• Historical presence sessions support

Completed: v1.8 - HTTPS OTA Support

• Support HTTPS URLs for OTA updates (esp_https_ota)
• CI: Upload firmware to Gitea releases for OTA
• CI: Simplify deploy script to pure POSIX sh

Completed: v1.7 - Baseline Calibration & Presence Detection

• CALIBRATE command (capture N seconds of CSI, average per-subcarrier amplitudes)
• CALIBRATE STATUS / CALIBRATE CLEAR subcommands
• Presence scoring (normalized Euclidean distance vs baseline, rolling window of 50)
• PRESENCE ON/OFF command (NVS persisted, requires valid baseline)
• PRESENCE THRESHOLD command (0.001-1.0, NVS persisted, default 0.05)
• Presence events (EVENT,<hostname>,presence=<0|1> score=<float>) with 10s holdoff
• Calibration done event (EVENT,<hostname>,calibrate=done packets=<n> nsub=<n>)
• presence= and pr_score= fields in STATUS reply
• NVS persistence: bl_amps (blob), bl_nsub (i8), presence (i8), pr_thresh (i32)
• config_save_blob / config_erase_key NVS helpers
• n_sub field in csi_features_t, amps_out parameter in csi_extract_features
• Pi-side: parse presence/calibrate events in watch daemon
• Pi-side: store in sensor_events table (esp-ctl osint events)
• Pi-side: HA webhooks for presence_change and calibrate_done
• Fleet OTA to v1.7, calibrated hollow-acorn + amber-maple

Completed: v1.6 - Power Management

• ESP-IDF power management framework (DFS 240/80 MHz + light sleep)
• sdkconfig: CONFIG_PM_ENABLE, CONFIG_FREERTOS_USE_TICKLESS_IDLE
• POWERSAVE command (WiFi modem sleep toggle, NVS persisted, default off)
• powersave= field in STATUS reply
• POWERTEST save/restore of powersave state
• Deployed to fleet, NVS persistence verified

Completed: v1.5 - Event Handling & NVS Persistence

• EVENT packet parsing in watch daemon (motion, wifi_reconnect, powertest)
• Sensor heartbeat tracking (offline/online detection, configurable timeout)
• sensor_events table with indexed queries (esp-ctl osint events)
• HA webhooks for sensor_offline, sensor_online, motion_change
• NVS persistence for SCANRATE and PROBERATE commands
• Sensor events count in esp-ctl osint stats

Completed: v1.4 - Multi-Sensor & Validation

• Multi-sensor BLE correlation in esp-ctl (zone tracking by source sensor)
• Zone tracking with EMA RSSI (esp-ctl osint zones, device_zones table)
• Per-sensor breakdown in MAC profile (esp-ctl osint mac)
• POWERTEST command (7-phase power profiling with EVENT markers)
• Parallel OTA fleet updates (esp-fleet ota --parallel)

Completed: v1.3 - Security & OSINT

• HMAC command authentication (firmware + esp-ctl/esp-cmd/esp-fleet/esp-ota)
• AUTH command (set/query/disable secret, NVS persisted)
• auth=on/off in STATUS
• Deauth flood detection (ring buffer, aggregate ALERT_DATA)
• FLOODTHRESH command (count + window, NVS persisted)
• flood_thresh field in STATUS
• MAC OUI vendor lookup (esp-ctl oui, IEEE CSV database)
• OSINT SQLite database (probe_ssids, device_sightings tables)
• Watch daemon (esp-ctl watch — listen + enrich + store)
• OSINT query CLI (esp-ctl osint probes/devices/mac/stats)
• Home Assistant webhook integration (deauth_flood, unknown_probe, unknown_ble)
• Watch config file (~/.config/esp-ctl/watch.yaml)

Completed: v1.2

• On-device CSI feature extraction (amp_rms, amp_std, amp_max, amp_max_idx, energy)
• CSIMODE command: RAW, COMPACT, HYBRID N (NVS persisted)
• Compact payload format "F:rms,std,max,idx,energy" (~80% bandwidth reduction)
• Hybrid mode: compact every packet, raw every Nth
• STATUS fields: csi_mode, hybrid_n
• Adaptive sampling reuses extracted energy (no duplicate computation in COMPACT/HYBRID)

Completed: v1.1

• Sensor ID in data packets (hostname prefix on CSI_DATA, BLE_DATA, EVENT)
• Deauth/disassoc frame detection (ALERT_DATA via promiscuous mode)
• Chip temperature reporting in STATUS reply
• BLE alerting (esp-ctl ble --known, alert on unknown MACs)
• BLE dwell time tracking (dwell column in --track summary)
• Timestamped event logging (esp-ctl listen --timestamp)
• Alert filter in esp-ctl (listen -f alert)
• Runtime HOSTNAME command (NVS persisted, mDNS updated)
• WiFi probe request capture (PROBE_DATA via promiscuous mode, 10s dedup)
• mDNS service advertisement (_esp-csi._udp)
• mDNS sensor discovery (esp-ctl discover)
• Probe filter in esp-ctl (listen -f probe)
• OTA fleet to same firmware (ca526ef)
• CSI packet counter in STATUS (csi_count=)
• Raw uptime in STATUS (uptime_s=)
• WiFi reconnect EVENT emission
• SCANRATE command (BLE scan interval tuning, 5-300s)
• PROBERATE command (probe dedup cooldown tuning, 1-300s)
• Fix: promiscuous mode disables CSI on original ESP32 — guarded with #if
• FreeRTOS CPU runtime stats in PROFILE (trace facility enabled)
• Increased cmd_task stack 4KB → 6KB (was 516 bytes free)
• WiFi channel in STATUS (channel=)
• Boot counter in NVS (boots=)
• RSSI min/max tracking from CSI frames (rssi_min=, rssi_max=)
• Actual CSI rate in STATUS (csi_rate=)

Completed: v0.5 - BLE Scanning

• Enable Bluetooth alongside WiFi (NimBLE, BLE ON/OFF command)
• Periodic BLE advertisement scanning
• Report device MAC, RSSI, name via UDP
• Pi-side BLE device tracking (esp-ctl ble --track)
• PROFILE command (heap, stack watermarks, CPU runtime stats)
• TARGET command (runtime UDP destination config)

Completed: v0.4 - Adaptive Sampling

• On-device CSI wander calculation (coefficient of variation)
• Adaptive rate: 10 pkt/s idle (3s holdoff) → 100 pkt/s on motion
• EVENT notification to Pi on rate change
• ADAPTIVE ON/OFF command (NVS persisted)
• THRESHOLD command for tuning sensitivity (NVS persisted)
• RATE command disables adaptive mode
• adaptive/motion fields in STATUS reply
• OTA deployed and verified on amber-maple

Completed: v0.3 - OTA Updates

• Dual OTA partition table (partitions.csv)
• 4MB flash, custom partitions, rollback in sdkconfig.defaults
• Firmware: OTA command, ota_task, LED_OTA, rollback validation
• Firmware: version in STATUS reply
• Pi-side esp-ota tool (HTTP server + OTA orchestration)
• esp-fleet ota subcommand (sequential fleet update)
• Build and USB-flash amber-maple (partition table change)
• End-to-end OTA test verified
• Regenerate sdkconfig.sample
• Update CHEATSHEET.md, USAGE.md

Completed: v0.2 - Remote Management

• Firmware: UDP command listener (port 5501)
• Firmware: LED status indicator (GPIO2)
• Firmware: NVS config persistence (rate, tx_power)
• Firmware: REBOOT, IDENTIFY, STATUS commands
• Firmware: RATE command (10-100 Hz, restarts ping)
• Firmware: POWER command (2-20 dBm)
• Pi-side: esp-cmd CLI tool
• Pi-side: esp-fleet fleet management tool
• mDNS hostname, watchdog, human-readable uptime

Completed: v0.1 - Documentation

• Copy firmware sources to project
• Document current firmware and settings
• Document build & flash workflow
• Create .gitignore for build artifacts
• Test building firmware from this repo
• Document CSI config options

Notes

• Adaptive threshold varies by environment; 0.001-0.01 is a good starting range
• NVS keys (24 total): send_rate, tx_power, adaptive, threshold, ble_scan, target_ip, target_port, hostname, boot_count, csi_mode, hybrid_n, auth_secret, flood_thresh, flood_window, scan_rate, probe_rate, powersave, presence, pr_thresh, bl_nsub, bl_amps, chanscan, chanscan_int
• EVENT packets include sensor hostname: EVENT,<hostname>,motion=... rate=... wander=...
• ALERT_DATA format: ALERT_DATA,<hostname>,<deauth|disassoc>,<sender_mac>,<target_mac>,<rssi> or ALERT_DATA,<hostname>,deauth_flood,<count>,<window_s>
• STATUS fields: uptime=, uptime_s=, heap=, rssi=, channel=, tx_power=, rate=, csi_rate=, hostname=, version=, adaptive=, motion=, ble=, target=, temp=, csi_count=, boots=, rssi_min=, rssi_max=, csi_mode=, hybrid_n=, auth=, flood_thresh=, powersave=, presence=, pr_score=
• PROBE_DATA format: PROBE_DATA,<hostname>,<mac>,<rssi>,<ssid>
• Probe requests deduped per MAC (default 10s cooldown, tunable via PROBERATE)
• mDNS service: _esp-csi._udp on data port (for sensor discovery)
• HOSTNAME command: HOSTNAME <name> sets NVS + mDNS, HOSTNAME queries current
• ESP32 limitation: Promiscuous mode (deauth/probe detection) disabled — breaks CSI. Works on ESP32-C6+.