feat: v0.1.4 — device intelligence dashboard

Add tabbed dashboard at /dashboard/ with three D3.js visualizations:
- Vendor treemap (devices grouped by type and vendor)
- SSID social graph (force-directed, shared probed SSIDs as edges)
- Fingerprint clusters (packed circles by device behavior)

Intelligence API endpoints at /api/v1/intelligence/ with param
validation. Dashboard built on htmx + Pico CSS dark theme + D3 v7,
all vendored locally (make vendor). 13 new tests (59 total).
This commit is contained in:
user
2026-02-06 18:59:53 +01:00
parent c1f580ba16
commit dfbd2a2196
27 changed files with 1177 additions and 15 deletions

View File

@@ -2,6 +2,22 @@
All notable changes to this project will be documented in this file.
## [v0.1.4] - 2026-02-06
### Added
- Device Intelligence dashboard at `/dashboard/` (htmx + Pico CSS + D3.js)
- Vendor treemap visualization (devices grouped by type and vendor)
- SSID social graph (force-directed graph linking devices by shared probed SSIDs)
- Fingerprint clusters (packed circles grouping devices by behavior)
- Intelligence API endpoints:
- `GET /api/v1/intelligence/vendor-treemap`
- `GET /api/v1/intelligence/ssid-graph`
- `GET /api/v1/intelligence/fingerprint-clusters`
- Vendored static assets: Pico CSS, htmx, D3.js v7 (`make vendor`)
- Jinja2 base template with dark theme
- Dashboard and API tests (13 new tests)
- Pagination totals, request logging, data retention CLI
## [v0.1.3] - 2026-02-05
### Added

View File

@@ -9,6 +9,8 @@ RUN pip install --no-cache-dir .
# Copy source
COPY src/ src/
COPY migrations/ migrations/
COPY static/ static/
COPY templates/ templates/
# Expose ports (TCP for HTTP, UDP for collector)
EXPOSE 5500/tcp

View File

@@ -1,4 +1,4 @@
.PHONY: help build run dev stop logs test migrate clean install start restart status oui cleanup
.PHONY: help build run dev stop logs test migrate clean install start restart status oui cleanup vendor
APP_NAME := esp32-web
PORT := 5500
@@ -115,6 +115,12 @@ container-stop:
container-logs:
podman logs -f $(APP_NAME)
vendor:
mkdir -p static/css/vendor static/js/vendor static/js/viz
curl -sLo static/css/vendor/pico.min.css https://cdn.jsdelivr.net/npm/@picocss/pico@2/css/pico.min.css
curl -sLo static/js/vendor/htmx.min.js https://unpkg.com/htmx.org@2.0.4/dist/htmx.min.js
curl -sLo static/js/vendor/d3.min.js https://cdn.jsdelivr.net/npm/d3@7/dist/d3.min.js
clean:
rm -rf __pycache__ .pytest_cache .ruff_cache
find . -type d -name __pycache__ -exec rm -rf {} +

View File

@@ -29,16 +29,26 @@
- [x] Sensor config endpoint (GET/PUT)
- [x] OTA trigger endpoint
- [x] Calibration trigger endpoint
- [ ] Sensor history/metrics (moved to v0.1.4)
- [ ] Sensor history/metrics (moved to v0.1.5)
## v0.1.4 - Zones & Presence
## v0.1.4 - Device Intelligence Dashboard [DONE]
- [x] Base dashboard layout (htmx + Pico CSS + D3.js dark theme)
- [x] Vendor treemap (D3 treemap by device type and vendor)
- [x] SSID social graph (D3 force-directed, shared probed SSIDs as edges)
- [x] Device fingerprint clusters (D3 packed circles by behavior)
- [x] Intelligence API endpoints (3 endpoints)
- [x] Vendored static assets (`make vendor`)
- [x] Pagination totals, request logging, data retention
## v0.1.5 - Zones & Presence
- [ ] Zone management (assign sensors to areas)
- [ ] Device zone tracking
- [ ] Dwell time analysis
- [ ] Presence history
## v0.1.5 - Production Ready
## v0.1.6 - Production Ready
- [ ] Authentication (API keys or JWT)
- [ ] Rate limiting
@@ -46,10 +56,22 @@
- [ ] Podman container deployment (quadlet/systemd unit)
- [ ] Production deployment guide
## v0.2.0 - Visualization Dashboard
- [x] Base dashboard layout (htmx + Pico CSS + D3.js) — done in v0.1.4
- [x] Vendor treemap — done in v0.1.4
- [x] SSID social graph — done in v0.1.4
- [x] Device fingerprint clusters — done in v0.1.4
- [ ] Presence timeline (device enter/leave Gantt chart)
- [ ] Deauth attack timeline (alert overlay with source/target)
## Future
- RSSI heatmap / triangulation
- CSI radar display
- Temporal knowledge graph
- Entropy dashboard (ambient awareness metric)
- WebSocket for real-time updates
- Web dashboard (htmx + Pico CSS)
- Home Assistant integration
- Grafana dashboards
- Webhook callbacks for alerts

View File

@@ -1,8 +1,8 @@
# ESP32-Web Tasks
**Last Updated:** 2026-02-05
**Last Updated:** 2026-02-06
## Current Sprint: v0.1.4 — Zones & Presence
## Current Sprint: v0.1.5 — Zones & Presence
### P1 - High
- [ ] Zone model (name, description, location)
@@ -16,9 +16,20 @@
- [ ] Dwell time analysis
- [ ] Presence history endpoint
### P3 - Low (carried from v0.1.3)
- [ ] Add pagination to all list endpoints
- [ ] Add request logging middleware
### P2 - Dashboard (v0.2.0)
- [ ] Presence timeline (Gantt chart, low effort)
- [ ] Deauth attack timeline (alert overlay, low effort)
## Completed: v0.1.4 — Device Intelligence Dashboard
- [x] Base dashboard layout (htmx + Pico CSS + D3.js dark theme)
- [x] Vendor treemap visualization (`/api/v1/intelligence/vendor-treemap`)
- [x] SSID social graph visualization (`/api/v1/intelligence/ssid-graph`)
- [x] Fingerprint clusters visualization (`/api/v1/intelligence/fingerprint-clusters`)
- [x] Jinja2 base template with tab navigation
- [x] Vendored static assets: Pico CSS, htmx, D3.js v7 (`make vendor`)
- [x] Dashboard + intelligence API tests (13 new tests, 59 total)
- [x] Pagination totals, request logging, data retention CLI
## Completed: v0.1.3 — Fleet Management
@@ -63,3 +74,4 @@
- API listens on TCP 5500
- Commands sent to sensors on UDP 5501
- SQLite for dev, PostgreSQL for prod
- Dashboard at `/dashboard/` with htmx tab switching

53
TODO.md
View File

@@ -2,22 +2,25 @@
## API
- [ ] Pagination for all list endpoints
- [x] Pagination for all list endpoints (with total count)
- [x] Request logging middleware
- [x] Data retention policy (auto-cleanup old records)
- [ ] Filter by date range
- [ ] Sort options
- [ ] Rate limiting (flask-limiter)
- [ ] API authentication (JWT or API keys)
- [ ] Request logging middleware
## OSINT
- [ ] Device fingerprinting by advertisement patterns
- [ ] SSID categorization (home, corporate, mobile hotspot)
- [ ] MAC randomization detection (correlate probe bursts, RSSI, timing)
- [ ] Device reputation scoring (randomized MAC, probe hygiene, visit frequency)
- [ ] Organizational mapping (group devices by vendor + behavior)
## Collector
- [ ] CSI data storage (optional, high volume)
- [ ] Data retention policy (auto-cleanup old records)
## Fleet Management
@@ -38,6 +41,50 @@
- [ ] Integration tests with mock sensors
- [ ] Load testing
## Visualizations
### Spatial / RF (D3.js)
- [ ] RSSI heatmap — triangulate device positions from multi-sensor readings, animate over time
- [ ] Sensor coverage Voronoi — show reach/overlap/blind spots
- [ ] Channel utilization spectrogram — waterfall display per sensor
### Device Intelligence
- [x] Device fingerprint clusters — group by behavior (probes, BLE company, cadence)
- [x] SSID social graph — devices as nodes, shared probed SSIDs as edges (reveals co-location history)
- [ ] Probe request worldmap — map probed SSIDs to geolocations via WiGLE
- [x] Vendor treemap — OUI + BLE company breakdown, anomaly spotting
### Temporal
- [ ] Presence timeline / Gantt — per-device strips showing enter/leave range (routines, anomalies)
- [ ] First-seen drift — highlight novel devices vs. known regulars
- [ ] Dwell time distributions — histogram, bimodal = passers-by vs. occupants
### Purple Team
- [ ] Deauth attack timeline — overlay alerts with source/target, correlate with device disappearances
- [ ] Evil twin detection — flag when probed SSID appears as local AP
- [ ] Flood intensity gauge — real-time deauth rate + historical sparklines
- [ ] Attack surface dashboard — broadcast probes (evil twin targets), static MACs (trackable), deauth-vulnerable
- [ ] Kill chain tracker — map events to MITRE ATT&CK for WiFi
### Experimental
- [ ] CSI radar — amplitude/phase matrix as real-time presence radar (if CSI enabled)
- [ ] Mesh consensus view — sensor agreement graph, fork/resolve visualization
- [ ] Temporal knowledge graph — devices/SSIDs/sensors/alerts with timestamped edges
- [ ] Adversarial simulation replay — VCR-style event playback with what-if scenarios
- [ ] Entropy dashboard — single ambient metric (new devices/hr, probe diversity, alert rate)
### Priority picks (high value, low-medium effort)
1. ~~Presence timeline (low effort, high value)~~ — next up
2. ~~Deauth attack timeline (low effort, high value)~~ — next up
3. ~~SSID social graph (medium effort, high value)~~ — done v0.1.4
4. ~~Device fingerprint clusters (medium effort, high value)~~ — done v0.1.4
5. RSSI heatmap / triangulation (high effort, very high value)
### Tech notes
- D3.js v7 + htmx + Pico CSS served locally from `static/vendor/`
- Dashboard at `/dashboard/` with htmx tab switching
- Intelligence API at `/api/v1/intelligence/*`
## Ideas
- WebSocket for live updates

View File

@@ -27,6 +27,12 @@ pytest -v # Verbose output
pytest -k test_sensors # Run specific tests
```
## Static Assets
```bash
make vendor # Download Pico CSS, htmx, D3.js to static/vendor/
```
## Container
```bash
@@ -66,6 +72,14 @@ curl localhost:5500/api/v1/probes/ssids
# Stats
curl localhost:5500/api/v1/stats
# Intelligence (Device Intelligence Dashboard)
curl localhost:5500/api/v1/intelligence/vendor-treemap
curl "localhost:5500/api/v1/intelligence/ssid-graph?hours=24&min_shared=1&limit=200"
curl "localhost:5500/api/v1/intelligence/fingerprint-clusters?hours=24"
# Dashboard
open http://localhost:5500/dashboard/
```
## Query Parameters
@@ -78,6 +92,8 @@ curl localhost:5500/api/v1/stats
| offset | devices, alerts, events, probes | Skip N results |
| ssid | probes | Filter by SSID |
| sensor_id | alerts, events | Filter by sensor |
| hours | intelligence/ssid-graph, intelligence/fingerprint-clusters | Time window (default: 24) |
| min_shared | intelligence/ssid-graph | Min shared SSIDs for link (default: 1) |
## Files

View File

@@ -20,7 +20,12 @@ def create_app(config_class=Config):
global _start_time
_start_time = datetime.now(UTC)
app = Flask(__name__)
project_root = Path(__file__).resolve().parent.parent.parent
app = Flask(
__name__,
static_folder=str(project_root / 'static'),
template_folder=str(project_root / 'templates'),
)
app.config.from_object(config_class)
# Initialize extensions
@@ -31,6 +36,9 @@ def create_app(config_class=Config):
from .api import bp as api_bp
app.register_blueprint(api_bp, url_prefix='/api/v1')
from .dashboard import bp as dashboard_bp
app.register_blueprint(dashboard_bp)
# Request logging
@app.before_request
def _start_timer():

View File

@@ -24,4 +24,4 @@ def paginate(query, schema_fn):
}
from . import sensors, devices, alerts, events, probes, stats, export # noqa: E402, F401
from . import sensors, devices, alerts, events, probes, stats, export, intelligence # noqa: E402, F401

View File

@@ -0,0 +1,241 @@
"""Device intelligence API endpoints."""
from collections import defaultdict
from datetime import datetime, UTC, timedelta
from flask import request
from . import bp
from ..extensions import db
from ..models import Device, Probe, Sighting
from ..utils.ble_companies import lookup_ble_company
@bp.route('/intelligence/vendor-treemap')
def vendor_treemap():
"""Return D3-ready hierarchy of devices grouped by type and vendor."""
# WiFi devices grouped by vendor
wifi_rows = db.session.execute(
db.select(
db.func.coalesce(Device.vendor, 'Unknown'),
db.func.count(),
)
.where(Device.device_type == 'wifi')
.group_by(Device.vendor)
).all()
# BLE devices grouped by company_id
ble_rows = db.session.execute(
db.select(
Device.company_id,
db.func.count(),
)
.where(Device.device_type == 'ble')
.group_by(Device.company_id)
).all()
wifi_children = [
{'name': vendor, 'value': count}
for vendor, count in wifi_rows if count > 0
]
ble_children = [
{'name': (lookup_ble_company(cid) or f'ID {cid}') if cid else 'Unknown', 'value': count}
for cid, count in ble_rows if count > 0
]
children = []
if wifi_children:
children.append({'name': 'wifi', 'children': wifi_children})
if ble_children:
children.append({'name': 'ble', 'children': ble_children})
return {'name': 'devices', 'children': children}
@bp.route('/intelligence/ssid-graph')
def ssid_graph():
"""Return force-graph data: nodes (devices) and links (shared probed SSIDs)."""
hours = max(1, request.args.get('hours', 24, type=int))
min_shared = max(1, request.args.get('min_shared', 1, type=int))
limit = max(1, min(request.args.get('limit', 200, type=int), 500))
cutoff = datetime.now(UTC) - timedelta(hours=hours)
# Get distinct (device_id, ssid) pairs in time window
rows = db.session.execute(
db.select(Probe.device_id, Probe.ssid)
.where(Probe.timestamp >= cutoff)
.distinct()
).all()
if not rows:
return {'nodes': [], 'links': [], 'ssids': []}
# Build mappings
device_ssids: dict[int, set[str]] = defaultdict(set)
ssid_devices: dict[str, set[int]] = defaultdict(set)
for device_id, ssid in rows:
device_ssids[device_id].add(ssid)
ssid_devices[ssid].add(device_id)
# Rank devices by probe diversity, cap at limit
top_devices = sorted(device_ssids.keys(),
key=lambda d: len(device_ssids[d]),
reverse=True)[:limit]
top_set = set(top_devices)
# Find device pairs sharing >= min_shared SSIDs
links = []
seen_pairs: set[tuple[int, int]] = set()
for ssid, devices in ssid_devices.items():
device_list = [d for d in devices if d in top_set]
for i, d1 in enumerate(device_list):
for d2 in device_list[i + 1:]:
pair = (min(d1, d2), max(d1, d2))
if pair not in seen_pairs:
seen_pairs.add(pair)
shared = device_ssids[d1] & device_ssids[d2]
if len(shared) >= min_shared:
links.append({
'source_id': pair[0],
'target_id': pair[1],
'shared_ssids': sorted(shared),
'weight': len(shared),
})
# Collect device IDs that appear in at least one link
linked_ids = set()
for link in links:
linked_ids.add(link['source_id'])
linked_ids.add(link['target_id'])
# Also include isolated nodes from top devices
node_ids = linked_ids | top_set
# Fetch device details
devices = db.session.scalars(
db.select(Device).where(Device.id.in_(node_ids))
).all()
device_map = {d.id: d for d in devices}
nodes = []
for did in node_ids:
d = device_map.get(did)
if d:
nodes.append({
'id': d.mac,
'device_id': d.id,
'vendor': d.vendor or 'Unknown',
'type': d.device_type,
'ssid_count': len(device_ssids.get(did, set())),
})
# Map device IDs to MACs in links
out_links = []
for link in links:
src = device_map.get(link['source_id'])
tgt = device_map.get(link['target_id'])
if src and tgt:
out_links.append({
'source': src.mac,
'target': tgt.mac,
'shared_ssids': link['shared_ssids'],
'weight': link['weight'],
})
# SSID summary
ssid_summary = sorted(
[{'ssid': s, 'device_count': len(ds)} for s, ds in ssid_devices.items()],
key=lambda x: x['device_count'],
reverse=True,
)[:50]
return {'nodes': nodes, 'links': out_links, 'ssids': ssid_summary}
@bp.route('/intelligence/fingerprint-clusters')
def fingerprint_clusters():
"""Group active devices by behavior: (type, vendor, activity level)."""
hours = max(1, request.args.get('hours', 24, type=int))
cutoff = datetime.now(UTC) - timedelta(hours=hours)
# Probe counts per device
probe_counts = dict(db.session.execute(
db.select(Probe.device_id, db.func.count())
.where(Probe.timestamp >= cutoff)
.group_by(Probe.device_id)
).all())
# Sighting counts and avg RSSI per device
sighting_stats = {
row[0]: (row[1], row[2])
for row in db.session.execute(
db.select(
Sighting.device_id,
db.func.count(),
db.func.avg(Sighting.rssi),
)
.where(Sighting.timestamp >= cutoff)
.group_by(Sighting.device_id)
).all()
}
# Get all active device IDs
active_ids = set(probe_counts.keys()) | set(sighting_stats.keys())
if not active_ids:
return {'clusters': [], 'total_devices': 0, 'total_clusters': 0}
# Fetch devices
devices = db.session.scalars(
db.select(Device).where(Device.id.in_(active_ids))
).all()
# Bucket activity level
def activity_bucket(count: int) -> str:
if count > 20:
return 'High'
if count >= 5:
return 'Medium'
return 'Low'
# Group by (device_type, vendor, activity_bucket)
clusters: dict[tuple[str, str, str], list] = defaultdict(list)
for d in devices:
pc = probe_counts.get(d.id, 0)
sc, avg_rssi = sighting_stats.get(d.id, (0, None))
total_activity = pc + sc
bucket = activity_bucket(total_activity)
vendor = d.vendor or 'Unknown'
key = (d.device_type, vendor, bucket)
clusters[key].append({
'mac': d.mac,
'probe_count': pc,
'sighting_count': sc,
'avg_rssi': round(avg_rssi) if avg_rssi is not None else None,
})
# Build response
result = []
for idx, ((dtype, vendor, bucket), devs) in enumerate(
sorted(clusters.items(), key=lambda x: len(x[1]), reverse=True)
):
probe_total = sum(d['probe_count'] for d in devs)
sight_total = sum(d['sighting_count'] for d in devs)
rssi_vals = [d['avg_rssi'] for d in devs if d['avg_rssi'] is not None]
result.append({
'id': idx,
'label': f'{vendor} {dtype.upper()} - {bucket} Activity',
'device_type': dtype,
'vendor': vendor,
'activity': bucket,
'device_count': len(devs),
'devices': devs,
'centroid': {
'probe_rate': round(probe_total / len(devs), 1) if devs else 0,
'sighting_rate': round(sight_total / len(devs), 1) if devs else 0,
'avg_rssi': round(sum(rssi_vals) / len(rssi_vals)) if rssi_vals else None,
},
})
return {
'clusters': result,
'total_devices': len(active_ids),
'total_clusters': len(result),
}

View File

@@ -0,0 +1,40 @@
"""Dashboard blueprint."""
from flask import Blueprint, render_template
from ..extensions import db
from ..models import Device, Sensor
bp = Blueprint('dashboard', __name__, url_prefix='/dashboard')
@bp.route('/')
def index():
"""Render the main dashboard page."""
total_devices = db.session.scalar(
db.select(db.func.count()).select_from(Device)
)
total_sensors = db.session.scalar(
db.select(db.func.count()).select_from(Sensor)
)
return render_template(
'dashboard/index.html',
total_devices=total_devices,
total_sensors=total_sensors,
)
@bp.route('/tab/vendor-treemap')
def tab_vendor_treemap():
"""Return vendor treemap partial."""
return render_template('dashboard/partials/vendor_treemap.html')
@bp.route('/tab/ssid-graph')
def tab_ssid_graph():
"""Return SSID graph partial."""
return render_template('dashboard/partials/ssid_graph.html')
@bp.route('/tab/fingerprint-clusters')
def tab_fingerprint_clusters():
"""Return fingerprint clusters partial."""
return render_template('dashboard/partials/fingerprint_clusters.html')

149
static/css/main.css Normal file
View File

@@ -0,0 +1,149 @@
/* ESP32-Web Dashboard — dark theme overrides for Pico CSS */
:root {
--pico-font-family: system-ui, -apple-system, sans-serif;
}
/* Tab navigation */
.tab-nav {
display: flex;
gap: 0;
border-bottom: 2px solid var(--pico-muted-border-color);
margin-bottom: 1.5rem;
}
.tab-nav a {
padding: 0.6rem 1.2rem;
text-decoration: none;
color: var(--pico-muted-color);
border-bottom: 2px solid transparent;
margin-bottom: -2px;
transition: color 0.2s, border-color 0.2s;
}
.tab-nav a:hover {
color: var(--pico-primary);
}
.tab-nav a.active {
color: var(--pico-primary);
border-bottom-color: var(--pico-primary);
}
/* Viz containers */
.viz-container {
position: relative;
width: 100%;
min-height: 400px;
}
.viz-container svg {
width: 100%;
height: 100%;
}
/* Loading state */
.viz-loading {
display: flex;
align-items: center;
justify-content: center;
min-height: 400px;
color: var(--pico-muted-color);
}
/* Treemap */
.treemap-cell {
stroke: var(--pico-background-color);
stroke-width: 1px;
}
.treemap-label {
fill: #fff;
font-size: 11px;
pointer-events: none;
}
/* Force graph */
.graph-node {
stroke: var(--pico-background-color);
stroke-width: 1.5px;
cursor: grab;
}
.graph-node:active {
cursor: grabbing;
}
.graph-link {
stroke: var(--pico-muted-border-color);
stroke-opacity: 0.5;
}
.graph-label {
font-size: 10px;
fill: var(--pico-muted-color);
pointer-events: none;
}
/* Pack layout */
.cluster-circle {
stroke: var(--pico-muted-border-color);
stroke-width: 1px;
fill-opacity: 0.7;
}
.cluster-label {
font-size: 11px;
fill: #fff;
pointer-events: none;
text-anchor: middle;
}
/* D3 tooltip */
.d3-tooltip {
position: absolute;
padding: 0.5rem 0.75rem;
background: var(--pico-card-background-color, #1a1a2e);
border: 1px solid var(--pico-muted-border-color);
border-radius: 4px;
font-size: 0.8rem;
color: var(--pico-color);
pointer-events: none;
z-index: 10;
max-width: 300px;
}
/* Stats row */
.stats-row {
display: flex;
gap: 1rem;
flex-wrap: wrap;
margin-bottom: 1.5rem;
}
.stat-card {
flex: 1;
min-width: 120px;
text-align: center;
padding: 1rem;
}
.stat-card .stat-value {
font-size: 2rem;
font-weight: 700;
color: var(--pico-primary);
}
.stat-card .stat-label {
font-size: 0.85rem;
color: var(--pico-muted-color);
}
/* Empty state */
.viz-empty {
display: flex;
align-items: center;
justify-content: center;
min-height: 300px;
color: var(--pico-muted-color);
font-style: italic;
}

4
static/css/vendor/pico.min.css vendored Normal file

File diff suppressed because one or more lines are too long

58
static/js/main.js Normal file
View File

@@ -0,0 +1,58 @@
/* ESP32-Web Dashboard — tab switching & shared utilities */
document.addEventListener('DOMContentLoaded', () => {
initTabs();
});
function initTabs() {
const tabs = document.querySelectorAll('.tab-nav a');
tabs.forEach(tab => {
tab.addEventListener('click', (e) => {
e.preventDefault();
if (tab.classList.contains('active')) return;
tabs.forEach(t => t.classList.remove('active'));
tab.classList.add('active');
// htmx handles the AJAX load via hx-get on the element
});
});
// Load first tab on page load
const first = document.querySelector('.tab-nav a');
if (first) {
first.click();
}
}
/* Shared D3 tooltip */
function createTooltip() {
let tip = document.querySelector('.d3-tooltip');
if (!tip) {
tip = document.createElement('div');
tip.className = 'd3-tooltip';
tip.style.display = 'none';
document.body.appendChild(tip);
}
return {
show(html, event) {
tip.innerHTML = html;
tip.style.display = 'block';
tip.style.left = (event.pageX + 12) + 'px';
tip.style.top = (event.pageY - 12) + 'px';
},
move(event) {
tip.style.left = (event.pageX + 12) + 'px';
tip.style.top = (event.pageY - 12) + 'px';
},
hide() {
tip.style.display = 'none';
}
};
}
/* Responsive SVG dimensions from container */
function getVizSize(container) {
return {
width: Math.max(container.clientWidth || container.parentElement.clientWidth, 300),
height: Math.max(container.clientHeight, 400)
};
}

2
static/js/vendor/d3.min.js vendored Normal file

File diff suppressed because one or more lines are too long

1
static/js/vendor/htmx.min.js vendored Normal file

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,110 @@
/* Fingerprint Clusters — D3 packed circles grouping devices by behavior */
function renderFingerprintClusters(selector, apiUrl) {
const container = document.querySelector(selector);
const tooltip = createTooltip();
const { width, height } = getVizSize(container);
fetch(apiUrl + '?hours=24')
.then(r => r.json())
.then(data => {
if (!data.clusters || data.clusters.length === 0) {
container.innerHTML = '<div class="viz-empty">No active devices in the last 24 hours</div>';
return;
}
// Build hierarchy for d3.pack
const hierarchy = {
name: 'root',
children: data.clusters.map(c => ({
name: c.label,
value: c.device_count,
cluster: c,
})),
};
const activityColor = d3.scaleOrdinal()
.domain(['Low', 'Medium', 'High'])
.range(['#495057', '#f59f00', '#ff6b6b']);
const root = d3.hierarchy(hierarchy)
.sum(d => d.value || 0)
.sort((a, b) => b.value - a.value);
d3.pack()
.size([width, height])
.padding(8)(root);
const svg = d3.select(selector)
.append('svg')
.attr('viewBox', `0 0 ${width} ${height}`)
.attr('preserveAspectRatio', 'xMidYMid meet');
const leaves = svg.selectAll('g')
.data(root.leaves())
.join('g')
.attr('transform', d => `translate(${d.x},${d.y})`);
leaves.append('circle')
.attr('class', 'cluster-circle')
.attr('r', d => d.r)
.attr('fill', d => activityColor(d.data.cluster.activity))
.on('mouseover', (event, d) => {
const c = d.data.cluster;
tooltip.show(
`<strong>${c.label}</strong><br>` +
`Devices: ${c.device_count}<br>` +
`Avg probe rate: ${c.centroid.probe_rate}/device<br>` +
`Avg sighting rate: ${c.centroid.sighting_rate}/device<br>` +
`Avg RSSI: ${c.centroid.avg_rssi !== null ? c.centroid.avg_rssi + ' dBm' : 'N/A'}`,
event
);
d3.select(event.target).attr('fill-opacity', 1).attr('stroke', '#fff');
})
.on('mousemove', (event) => {
tooltip.move(event);
})
.on('mouseout', (event) => {
tooltip.hide();
d3.select(event.target).attr('fill-opacity', 0.7).attr('stroke', null);
});
// Labels for circles big enough
leaves.append('text')
.attr('class', 'cluster-label')
.attr('dy', '-0.3em')
.text(d => {
if (d.r < 25) return '';
const name = d.data.cluster.vendor;
const maxChars = Math.floor(d.r * 2 / 7);
return name.length > maxChars ? name.slice(0, maxChars - 1) + '\u2026' : name;
});
leaves.append('text')
.attr('class', 'cluster-label')
.attr('dy', '1em')
.style('font-size', '10px')
.style('opacity', 0.8)
.text(d => d.r >= 20 ? d.data.cluster.device_count : '');
// Legend
const legend = svg.append('g')
.attr('transform', `translate(${width - 140}, 20)`);
['Low', 'Medium', 'High'].forEach((level, i) => {
const g = legend.append('g')
.attr('transform', `translate(0, ${i * 22})`);
g.append('circle')
.attr('r', 6)
.attr('fill', activityColor(level));
g.append('text')
.attr('x', 14)
.attr('y', 4)
.attr('fill', '#adb5bd')
.style('font-size', '12px')
.text(level + ' Activity');
});
})
.catch(err => {
container.innerHTML = `<div class="viz-empty">Failed to load data: ${err.message}</div>`;
});
}

110
static/js/viz/ssid_graph.js Normal file
View File

@@ -0,0 +1,110 @@
/* SSID Social Graph — D3 force-directed graph linking devices by shared probed SSIDs */
function renderSsidGraph(selector, apiUrl) {
const container = document.querySelector(selector);
const tooltip = createTooltip();
const { width, height } = getVizSize(container);
fetch(apiUrl + '?hours=24&min_shared=1&limit=200')
.then(r => r.json())
.then(data => {
if (!data.nodes || data.nodes.length === 0) {
container.innerHTML = '<div class="viz-empty">No probe data available</div>';
return;
}
const color = d3.scaleOrdinal()
.domain(['wifi', 'ble'])
.range(['#4dabf7', '#69db7c']);
const sizeScale = d3.scaleSqrt()
.domain([1, d3.max(data.nodes, d => d.ssid_count) || 1])
.range([4, 16]);
const svg = d3.select(selector)
.append('svg')
.attr('viewBox', `0 0 ${width} ${height}`)
.attr('preserveAspectRatio', 'xMidYMid meet');
const simulation = d3.forceSimulation(data.nodes)
.force('link', d3.forceLink(data.links).id(d => d.id).distance(80))
.force('charge', d3.forceManyBody().strength(-100))
.force('center', d3.forceCenter(width / 2, height / 2))
.force('collision', d3.forceCollide().radius(d => sizeScale(d.ssid_count) + 2));
const link = svg.append('g')
.selectAll('line')
.data(data.links)
.join('line')
.attr('class', 'graph-link')
.attr('stroke-width', d => Math.max(1, d.weight));
const node = svg.append('g')
.selectAll('circle')
.data(data.nodes)
.join('circle')
.attr('class', 'graph-node')
.attr('r', d => sizeScale(d.ssid_count))
.attr('fill', d => color(d.type))
.on('mouseover', (event, d) => {
tooltip.show(
`<strong>${d.id}</strong><br>` +
`Vendor: ${d.vendor}<br>` +
`Type: ${d.type}<br>` +
`Probed SSIDs: ${d.ssid_count}`,
event
);
d3.select(event.target).attr('stroke', '#fff').attr('stroke-width', 3);
})
.on('mousemove', (event) => {
tooltip.move(event);
})
.on('mouseout', (event) => {
tooltip.hide();
d3.select(event.target).attr('stroke', null).attr('stroke-width', 1.5);
})
.call(drag(simulation));
// Link hover
link.on('mouseover', (event, d) => {
const ssids = d.shared_ssids.slice(0, 5).join(', ');
const more = d.shared_ssids.length > 5 ? ` (+${d.shared_ssids.length - 5} more)` : '';
tooltip.show(
`<strong>Shared SSIDs (${d.weight})</strong><br>${ssids}${more}`,
event
);
}).on('mousemove', (event) => tooltip.move(event))
.on('mouseout', () => tooltip.hide());
simulation.on('tick', () => {
link
.attr('x1', d => d.source.x)
.attr('y1', d => d.source.y)
.attr('x2', d => d.target.x)
.attr('y2', d => d.target.y);
node
.attr('cx', d => d.x)
.attr('cy', d => d.y);
});
function drag(sim) {
return d3.drag()
.on('start', (event, d) => {
if (!event.active) sim.alphaTarget(0.3).restart();
d.fx = d.x;
d.fy = d.y;
})
.on('drag', (event, d) => {
d.fx = event.x;
d.fy = event.y;
})
.on('end', (event, d) => {
if (!event.active) sim.alphaTarget(0);
d.fx = null;
d.fy = null;
});
}
})
.catch(err => {
container.innerHTML = `<div class="viz-empty">Failed to load data: ${err.message}</div>`;
});
}

View File

@@ -0,0 +1,95 @@
/* Vendor Treemap — D3 treemap of device vendors by type */
function renderVendorTreemap(selector, apiUrl) {
const container = document.querySelector(selector);
const tooltip = createTooltip();
const { width, height } = getVizSize(container);
fetch(apiUrl)
.then(r => r.json())
.then(data => {
if (!data.children || data.children.length === 0) {
container.innerHTML = '<div class="viz-empty">No device data available</div>';
return;
}
const color = d3.scaleOrdinal()
.domain(['wifi', 'ble'])
.range(['#4dabf7', '#69db7c']);
const root = d3.hierarchy(data)
.sum(d => d.value || 0)
.sort((a, b) => b.value - a.value);
d3.treemap()
.size([width, height])
.padding(2)
.round(true)(root);
const svg = d3.select(selector)
.append('svg')
.attr('viewBox', `0 0 ${width} ${height}`)
.attr('preserveAspectRatio', 'xMidYMid meet');
const leaves = svg.selectAll('g')
.data(root.leaves())
.join('g')
.attr('transform', d => `translate(${d.x0},${d.y0})`);
leaves.append('rect')
.attr('class', 'treemap-cell')
.attr('width', d => Math.max(0, d.x1 - d.x0))
.attr('height', d => Math.max(0, d.y1 - d.y0))
.attr('fill', d => {
const type = d.parent ? d.parent.data.name : 'wifi';
return color(type);
})
.attr('fill-opacity', 0.8)
.on('mouseover', (event, d) => {
const type = d.parent ? d.parent.data.name : '';
tooltip.show(
`<strong>${d.data.name}</strong><br>` +
`Type: ${type}<br>` +
`Devices: ${d.value}`,
event
);
d3.select(event.target).attr('fill-opacity', 1);
})
.on('mousemove', (event) => {
tooltip.move(event);
})
.on('mouseout', (event) => {
tooltip.hide();
d3.select(event.target).attr('fill-opacity', 0.8);
});
// Labels only for cells big enough
leaves.append('text')
.attr('class', 'treemap-label')
.attr('x', 4)
.attr('y', 14)
.text(d => {
const w = d.x1 - d.x0;
const h = d.y1 - d.y0;
if (w < 40 || h < 18) return '';
const name = d.data.name;
const maxChars = Math.floor(w / 7);
return name.length > maxChars ? name.slice(0, maxChars - 1) + '\u2026' : name;
});
leaves.append('text')
.attr('class', 'treemap-label')
.attr('x', 4)
.attr('y', 28)
.style('font-size', '10px')
.style('opacity', 0.8)
.text(d => {
const w = d.x1 - d.x0;
const h = d.y1 - d.y0;
if (w < 30 || h < 32) return '';
return d.value;
});
})
.catch(err => {
container.innerHTML = `<div class="viz-empty">Failed to load data: ${err.message}</div>`;
});
}

30
templates/base.html Normal file
View File

@@ -0,0 +1,30 @@
<!DOCTYPE html>
<html lang="en" data-theme="dark">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{% block title %}ESP32-Web{% endblock %}</title>
<link rel="stylesheet" href="{{ url_for('static', filename='css/vendor/pico.min.css') }}">
<link rel="stylesheet" href="{{ url_for('static', filename='css/main.css') }}">
{% block head %}{% endblock %}
</head>
<body>
<nav class="container-fluid">
<ul>
<li><strong>ESP32-Web</strong></li>
</ul>
<ul>
<li><a href="{{ url_for('dashboard.index') }}">Dashboard</a></li>
<li><a href="/docs">API Docs</a></li>
<li><a href="/health">Health</a></li>
</ul>
</nav>
<main class="container">
{% block content %}{% endblock %}
</main>
<script src="{{ url_for('static', filename='js/vendor/htmx.min.js') }}"></script>
<script src="{{ url_for('static', filename='js/vendor/d3.min.js') }}"></script>
<script src="{{ url_for('static', filename='js/main.js') }}"></script>
{% block scripts %}{% endblock %}
</body>
</html>

View File

@@ -0,0 +1,19 @@
{% extends "base.html" %}
{% block title %}Device Intelligence — ESP32-Web{% endblock %}
{% block content %}
<hgroup>
<h2>Device Intelligence</h2>
<p>{{ total_devices }} devices tracked across {{ total_sensors }} sensors</p>
</hgroup>
<nav class="tab-nav">
<a href="#" hx-get="{{ url_for('dashboard.tab_vendor_treemap') }}" hx-target="#tab-content">Vendor Treemap</a>
<a href="#" hx-get="{{ url_for('dashboard.tab_ssid_graph') }}" hx-target="#tab-content">SSID Graph</a>
<a href="#" hx-get="{{ url_for('dashboard.tab_fingerprint_clusters') }}" hx-target="#tab-content">Fingerprint Clusters</a>
</nav>
<div id="tab-content">
<div class="viz-loading" aria-busy="true">Loading...</div>
</div>
{% endblock %}

View File

@@ -0,0 +1,3 @@
<div class="viz-container" id="fingerprint-clusters"></div>
<script src="{{ url_for('static', filename='js/viz/fingerprint_clusters.js') }}"></script>
<script>renderFingerprintClusters('#fingerprint-clusters', '{{ url_for("api.fingerprint_clusters") }}');</script>

View File

@@ -0,0 +1,3 @@
<div class="viz-container" id="ssid-graph"></div>
<script src="{{ url_for('static', filename='js/viz/ssid_graph.js') }}"></script>
<script>renderSsidGraph('#ssid-graph', '{{ url_for("api.ssid_graph") }}');</script>

View File

@@ -0,0 +1,3 @@
<div class="viz-container" id="vendor-treemap"></div>
<script src="{{ url_for('static', filename='js/viz/vendor_treemap.js') }}"></script>
<script>renderVendorTreemap('#vendor-treemap', '{{ url_for("api.vendor_treemap") }}');</script>

View File

@@ -0,0 +1,136 @@
"""Intelligence API tests."""
from datetime import datetime, UTC, timedelta
from esp32_web.extensions import db
from esp32_web.models import Device, Probe, Sighting, Sensor
def _seed_devices(app):
"""Create test devices, probes, and sightings."""
with app.app_context():
sensor = Sensor(hostname='test-sensor', ip='192.168.1.1')
db.session.add(sensor)
db.session.flush()
d1 = Device(mac='aa:bb:cc:dd:ee:01', device_type='wifi',
vendor='Apple, Inc.', first_seen=datetime.now(UTC),
last_seen=datetime.now(UTC))
d2 = Device(mac='aa:bb:cc:dd:ee:02', device_type='wifi',
vendor='Samsung', first_seen=datetime.now(UTC),
last_seen=datetime.now(UTC))
d3 = Device(mac='aa:bb:cc:dd:ee:03', device_type='ble',
company_id=0x004C, first_seen=datetime.now(UTC),
last_seen=datetime.now(UTC))
db.session.add_all([d1, d2, d3])
db.session.flush()
now = datetime.now(UTC)
# Both d1 and d2 probe "HomeNet"
db.session.add_all([
Probe(device_id=d1.id, sensor_id=sensor.id, ssid='HomeNet',
rssi=-50, channel=6, timestamp=now),
Probe(device_id=d2.id, sensor_id=sensor.id, ssid='HomeNet',
rssi=-60, channel=6, timestamp=now),
Probe(device_id=d1.id, sensor_id=sensor.id, ssid='Office',
rssi=-55, channel=1, timestamp=now),
])
# Sightings
db.session.add_all([
Sighting(device_id=d1.id, sensor_id=sensor.id, rssi=-50, timestamp=now),
Sighting(device_id=d3.id, sensor_id=sensor.id, rssi=-70, timestamp=now),
])
db.session.commit()
# -- Vendor Treemap --
def test_vendor_treemap_empty(client):
"""Test treemap with no devices."""
response = client.get('/api/v1/intelligence/vendor-treemap')
assert response.status_code == 200
assert response.json['name'] == 'devices'
assert response.json['children'] == []
def test_vendor_treemap_with_data(client, app):
"""Test treemap returns grouped vendor data."""
_seed_devices(app)
response = client.get('/api/v1/intelligence/vendor-treemap')
assert response.status_code == 200
data = response.json
assert data['name'] == 'devices'
assert len(data['children']) > 0
# Find wifi group
wifi = next((c for c in data['children'] if c['name'] == 'wifi'), None)
assert wifi is not None
assert len(wifi['children']) >= 1
# Find ble group
ble = next((c for c in data['children'] if c['name'] == 'ble'), None)
assert ble is not None
assert len(ble['children']) >= 1
# -- SSID Graph --
def test_ssid_graph_empty(client):
"""Test graph with no probes."""
response = client.get('/api/v1/intelligence/ssid-graph')
assert response.status_code == 200
assert response.json['nodes'] == []
assert response.json['links'] == []
def test_ssid_graph_with_data(client, app):
"""Test graph returns nodes and links for shared SSIDs."""
_seed_devices(app)
response = client.get('/api/v1/intelligence/ssid-graph?hours=24')
assert response.status_code == 200
data = response.json
assert len(data['nodes']) >= 2
# d1 and d2 share "HomeNet" → at least one link
assert len(data['links']) >= 1
link = data['links'][0]
assert 'HomeNet' in link['shared_ssids']
assert link['weight'] >= 1
# SSID summary present
assert len(data['ssids']) >= 1
def test_ssid_graph_params(client, app):
"""Test graph with custom parameters."""
_seed_devices(app)
response = client.get('/api/v1/intelligence/ssid-graph?hours=1&min_shared=5&limit=10')
assert response.status_code == 200
# With min_shared=5, no pairs should match
assert response.json['links'] == []
# -- Fingerprint Clusters --
def test_fingerprint_clusters_empty(client):
"""Test clusters with no activity."""
response = client.get('/api/v1/intelligence/fingerprint-clusters')
assert response.status_code == 200
assert response.json['clusters'] == []
assert response.json['total_devices'] == 0
def test_fingerprint_clusters_with_data(client, app):
"""Test clusters group devices by behavior."""
_seed_devices(app)
response = client.get('/api/v1/intelligence/fingerprint-clusters?hours=24')
assert response.status_code == 200
data = response.json
assert data['total_devices'] >= 2
assert data['total_clusters'] >= 1
# Each cluster has expected structure
cluster = data['clusters'][0]
assert 'label' in cluster
assert 'device_count' in cluster
assert 'devices' in cluster
assert 'centroid' in cluster
assert 'probe_rate' in cluster['centroid']
assert 'sighting_rate' in cluster['centroid']

View File

View File

@@ -0,0 +1,29 @@
"""Dashboard view tests."""
def test_dashboard_index(client):
"""Test main dashboard page loads."""
response = client.get('/dashboard/')
assert response.status_code == 200
assert b'Device Intelligence' in response.data
def test_dashboard_tab_vendor_treemap(client):
"""Test vendor treemap partial loads."""
response = client.get('/dashboard/tab/vendor-treemap')
assert response.status_code == 200
assert b'vendor-treemap' in response.data
def test_dashboard_tab_ssid_graph(client):
"""Test SSID graph partial loads."""
response = client.get('/dashboard/tab/ssid-graph')
assert response.status_code == 200
assert b'ssid-graph' in response.data
def test_dashboard_tab_fingerprint_clusters(client):
"""Test fingerprint clusters partial loads."""
response = client.get('/dashboard/tab/fingerprint-clusters')
assert response.status_code == 200
assert b'fingerprint-clusters' in response.data