Compare commits

...

5 Commits

Author SHA1 Message Date
User
5fbf096a04 feat: integrate WebSocket with live tracking
- Add WebSocket state variables (wsEnabled, wsConnected)
- Add initWebSocket() function with connection and event handling
- Add handleWebSocketScanUpdate() for processing WS scan events
- Modify startLiveTracking() to use WS mode with HTTP fallback
- Update index.html to load socket.io and websocket.js scripts
- Show [WS] indicator in status when using WebSocket mode

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 04:56:10 +01:00
User
8f4fa4e186 feat: add WebSocket client with fallback
- Add Socket.IO client library (v4.7.5)
- Create RFMapperWS class with:
  - Automatic reconnection (5 attempts)
  - HTTP polling fallback
  - Event listener system for scanUpdate, connected, disconnected
  - Floor subscription support

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 04:55:55 +01:00
User
14757f2e57 feat: add SocketIO server integration
- Initialize SocketIO with threading mode
- Add WebSocket event handlers (connect, disconnect, subscribe_floor)
- Add broadcast_scan_update() function for pushing scan results
- Integrate broadcast with BT scan endpoint
- Update run_server() to use socketio.run()

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 04:55:44 +01:00
User
fed08aa6dd feat: add flask-socketio dependency
Add flask-socketio>=5.3.0 for WebSocket support in the web interface.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 04:55:28 +01:00
User
98e2c6fc42 docs: update project docs for v1.0.0 release
- PROJECT.md: Add peer sync feature description and architecture
- ROADMAP.md: Update to v1.0.0, mark peer sync complete
- TASKS.md: Add peer sync completed tasks, update sprint
- TODO.md: Add multi-scanner section, mark items done
- CHANGELOG.md: Add v0.4.0 (HA) and v1.0.0 (peer sync) entries

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 04:41:24 +01:00
11 changed files with 455 additions and 15 deletions

View File

@@ -6,6 +6,43 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
---
## [1.0.0] - 2026-02-01
### Added
- **Multi-Scanner Peer Sync** - Multiple scanner instances share device metadata
- Peer registration API (`/api/peers/register`)
- Bidirectional sync (`/api/sync/devices`)
- Background sync thread (30s default interval)
- Automatic mutual registration between peers
- **Source scanner tracking** - Synced devices retain original detector info
- Devices positioned relative to detecting scanner, not local scanner
- Moving local scanner doesn't affect synced device positions
- **Peer scanner markers** - Show peer scanners on 3D map (cyan icons)
- Scanner identity configuration (id, name, floor, position)
- Timestamp-based conflict resolution for sync
### Changed
- Device positions now use peer's current position (live lookup)
- Popup shows "Source: <scanner>" for remotely-synced devices
---
## [0.4.0] - 2026-02-01
### Added
- **Home Assistant Integration** - Webhook-based presence tracking
- Scan results webhook (`rf_mapper_scan`)
- New device alerts webhook (`rf_mapper_new_device`)
- Device departure webhook (`rf_mapper_device_gone`)
- Configurable timeout for departure detection
- Scanner identity (id, name, floor) in webhooks
- Absence checker background thread
---
## [0.3.0] - 2026-02-01
### Added

View File

@@ -32,6 +32,10 @@ Understanding the RF environment around you is useful for:
- **Auto-scan** - Scheduled background scanning
- **Data Export** - JSON scan history with timestamps
- **Home Assistant Integration** - Webhook-based presence tracking, new device alerts, departure notifications
- **Multi-Scanner Peer Sync** - Multiple scanner instances share device metadata automatically
- Bidirectional sync with timestamp-based conflict resolution
- Source scanner tracking (devices positioned relative to detecting scanner)
- Peer scanner markers on 3D map
## Architecture
@@ -74,6 +78,15 @@ Understanding the RF environment around you is useful for:
│ │ Webhook │ │ Webhook │ │ Webhook │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Peer Scanner Sync │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Scanner A │◄── sync ──►│ Scanner B │ │
│ │ (rpios) │ │ (grokbox) │ │
│ └──────────────┘ └──────────────┘ │
│ Shared: floors, labels, favorites, notes │
└─────────────────────────────────────────────────────────────┘
```
## Dependencies
@@ -131,9 +144,13 @@ gps:
longitude: 4.3978
scanner:
id: rpios
name: "RPi OS Scanner"
wifi_interface: wlan0
bt_scan_timeout: 10
path_loss_exponent: 2.5
sync_interval_seconds: 30
accept_registrations: true
building:
enabled: true

View File

@@ -1,6 +1,6 @@
# RF Mapper Roadmap
## Current Version: v0.3.0-dev
## Current Version: v1.0.0
---
@@ -30,7 +30,7 @@
---
## v0.3.0 - 3D Visualization (IN PROGRESS)
## v0.3.0 - 3D Visualization (COMPLETED)
- [x] MapLibre GL JS integration
- [x] 3D building extrusion
@@ -45,7 +45,21 @@
- [x] Position smoothing/averaging (statistical, 5-sample + stddev)
- [x] Floor persistence in SQLite database
- [x] Popup persistence during live updates
- [ ] Device trails/history visualization
- [x] Device trails/history visualization
---
## v1.0.0 - Multi-Scanner Peer Sync (COMPLETED)
- [x] Scanner identity configuration (id, name, floor, position)
- [x] Peer registration API (`/api/peers/register`)
- [x] Bidirectional device sync (`/api/sync/devices`)
- [x] Timestamp-based conflict resolution
- [x] Source scanner tracking for synced devices
- [x] Device positions relative to detecting scanner
- [x] Peer scanner markers on 3D map
- [x] Background sync thread with configurable interval
- [x] Automatic mutual registration between peers
---
@@ -95,7 +109,7 @@
---
## v1.0.0 - Production Ready
## v1.1.0 - Production Hardening
- [ ] Comprehensive test suite
- [ ] Performance optimization
@@ -110,7 +124,7 @@
## Future Ideas (v2.0+)
- [ ] Multiple scanner nodes (distributed scanning)
- [x] Multiple scanner nodes (distributed scanning) - Done in v1.0.0
- [ ] Mesh network visualization
- [ ] Spectrum analyzer integration
- [ ] RTL-SDR support for wider RF
@@ -128,4 +142,6 @@
|---------|------|------------|
| v0.1.0 | 2026-01 | Initial CLI scanner |
| v0.2.0 | 2026-01 | Web dashboard |
| v0.3.0 | TBD | 3D visualization |
| v0.3.0 | 2026-02 | 3D visualization, floor positioning |
| v0.4.0 | 2026-02 | Home Assistant integration |
| v1.0.0 | 2026-02 | Multi-scanner peer sync |

View File

@@ -1,7 +1,8 @@
# RF Mapper - Active Tasks
**Sprint:** v0.3.0 - 3D Visualization
**Sprint:** v1.1.0 - Production Hardening
**Updated:** 2026-02-01
**Current Version:** v1.0.0
---
@@ -43,6 +44,11 @@
| [x] | Document API endpoints | docs/API.md |
| [x] | Create CHEATSHEET.md | Quick reference guide |
| [x] | Home Assistant webhook integration | Scan results, new device, departure alerts |
| [x] | Multi-scanner peer sync | Bidirectional sync between scanner instances |
| [x] | Source scanner tracking | Synced devices positioned relative to source |
| [x] | Peer scanner markers | Show peer scanners on 3D map |
| [ ] | Unit test coverage | pytest for core modules |
| [ ] | Docker container | Containerized deployment |
---
@@ -92,6 +98,12 @@
| Device trails for moving devices | 2026-02-01 |
| Manual position override (drag-drop) | 2026-02-01 |
| Home Assistant webhook integration | 2026-02-01 |
| Multi-scanner peer sync | 2026-02-01 |
| Peer registration API | 2026-02-01 |
| Bidirectional device sync | 2026-02-01 |
| Source scanner tracking | 2026-02-01 |
| Peer scanner markers on 3D map | 2026-02-01 |
| v1.0.0 release | 2026-02-01 |
---
@@ -118,3 +130,8 @@
- Floor assignments persist in database across page reloads
- Popups stay open during live tracking updates
- Manual position override: drag floor-assigned device markers to set custom position
- **Peer Sync**: Multiple scanner instances can share device metadata
- Devices synced from peers retain source scanner info
- Positions calculated relative to detecting scanner (not local scanner)
- Peer scanners shown on 3D map as cyan markers
- Background sync every 30 seconds (configurable)

22
TODO.md
View File

@@ -1,6 +1,7 @@
# RF Mapper - TODO / Backlog
**Last Updated:** 2026-02-01
**Current Version:** v1.0.0
---
@@ -24,6 +25,23 @@
---
## Multi-Scanner / Peer Sync
- [x] Scanner identity configuration
- [x] Peer registration API
- [x] Bidirectional device sync
- [x] Timestamp-based conflict resolution
- [x] Source scanner tracking for synced devices
- [x] Peer scanner markers on map
- [x] Background sync thread
- [ ] WebSocket real-time sync (instead of polling)
- [ ] Automatic peer discovery via mDNS/Bonjour
- [ ] Sync RSSI history for trilateration
- [ ] Web UI for peer management
- [ ] Sync conflict resolution UI
---
## Scanning
- [ ] Support multiple WiFi interfaces
@@ -222,6 +240,10 @@
- [x] Popup persistence during live updates
- [x] Manual position override (drag-drop for floor-assigned devices)
- [x] Home Assistant webhook integration (scan, new device, departure)
- [x] Multi-scanner peer sync (v1.0.0)
- [x] Bidirectional device metadata sync
- [x] Source scanner tracking for synced devices
- [x] Peer scanner markers on 3D map
---

View File

@@ -31,6 +31,7 @@ classifiers = [
dependencies = [
"flask>=3.0.0",
"flask-socketio>=5.3.0",
"pyyaml>=6.0",
"bleak>=0.21.0",
"requests>=2.28.0",

View File

@@ -7,7 +7,8 @@ import time
from datetime import datetime
from pathlib import Path
from flask import Flask, jsonify, render_template, request
from flask import Flask, current_app, jsonify, render_template, request
from flask_socketio import SocketIO, emit
from ..scanner import RFScanner
from ..distance import estimate_distance
@@ -16,6 +17,29 @@ from ..bluetooth_identify import identify_single_device, identify_device
from ..database import DeviceDatabase, init_database, get_database
from ..homeassistant import HAWebhooks, HAWebhookConfig
# Module-level SocketIO instance
socketio = SocketIO()
def broadcast_scan_update(app: Flask, devices: list[dict], scan_type: str = "bluetooth"):
"""Broadcast scan results to all connected WebSocket clients."""
sio = app.config.get("SOCKETIO")
if not sio:
return
scanner_identity = app.config.get("SCANNER_IDENTITY", {})
sio.emit(
"scan_update",
{
"type": scan_type,
"timestamp": datetime.now().isoformat(),
"scanner_id": scanner_identity.get("id", "unknown"),
"devices": devices,
},
namespace="/ws/scan",
)
class AutoScanner:
"""Background scanner that runs periodic scans"""
@@ -213,6 +237,10 @@ def create_app(config: Config | None = None) -> Flask:
# Store config reference
app.config["RF_CONFIG"] = config
# Initialize SocketIO with threading mode (compatible with existing threads)
socketio.init_app(app, cors_allowed_origins="*", async_mode="threading")
app.config["SOCKETIO"] = socketio
# Data directory from config
app.config["DATA_DIR"] = config.get_data_dir()
app.config["DATA_DIR"].mkdir(parents=True, exist_ok=True)
@@ -320,6 +348,31 @@ def create_app(config: Config | None = None) -> Flask:
location_label=config.auto_scan.location_label
)
# ==================== WebSocket Event Handlers ====================
@socketio.on("connect", namespace="/ws/scan")
def ws_connect():
"""Handle client connection."""
scanner_id = current_app.config.get("SCANNER_IDENTITY", {}).get("id", "unknown")
emit("connected", {"scanner_id": scanner_id})
print(f"[WS] Client connected: {request.sid}")
@socketio.on("disconnect", namespace="/ws/scan")
def ws_disconnect():
"""Handle client disconnection."""
print(f"[WS] Client disconnected: {request.sid}")
@socketio.on("subscribe_floor", namespace="/ws/scan")
def ws_subscribe_floor(data):
"""Subscribe to floor-specific updates."""
from flask_socketio import join_room
floor = data.get("floor", "all")
join_room(f"floor_{floor}")
emit("subscribed", {"floor": floor})
# ==================== HTTP Routes ====================
@app.route("/")
def index():
"""Main dashboard page"""
@@ -1072,6 +1125,9 @@ def create_app(config: Config | None = None) -> Flask:
scan_type="bluetooth"
)
# Broadcast to WebSocket clients
broadcast_scan_update(current_app, response_data["bluetooth_devices"], "bluetooth")
return jsonify(response_data)
# ==================== Historical Data API ====================
@@ -1476,10 +1532,12 @@ def run_server(
if log_requests:
print(f"Request logging: ENABLED")
print(f"Log output: {config.get_data_dir() / 'logs'}")
print(f"WebSocket: ENABLED (namespace /ws/scan)")
print(f"{'='*60}")
print(f"Server running at: http://{host}:{port}")
print(f"Local access: http://localhost:{port}")
print(f"Network access: http://<your-ip>:{port}")
print(f"{'='*60}\n")
app.run(host=host, port=port, debug=debug)
# Use socketio.run() for WebSocket support
socketio.run(app, host=host, port=port, debug=debug, allow_unsafe_werkzeug=True)

View File

@@ -33,6 +33,10 @@ let liveTrackingEnabled = false;
let liveTrackingInterval = null;
const LIVE_TRACKING_INTERVAL_MS = 4000; // 4 seconds
// WebSocket state
let wsEnabled = true; // Try WebSocket first
let wsConnected = false;
// Statistical movement detection
const SAMPLE_HISTORY_SIZE = 5; // Number of samples to keep for averaging
const MOVEMENT_THRESHOLD = 1.5; // meters - movement must exceed this + stddev margin
@@ -134,6 +138,9 @@ document.addEventListener('DOMContentLoaded', () => {
map3dInitialized = true;
}, 100);
// Initialize WebSocket connection
initWebSocket();
// Start BT live tracking by default after a short delay
setTimeout(() => {
startLiveTracking();
@@ -141,6 +148,152 @@ document.addEventListener('DOMContentLoaded', () => {
}, 2000);
});
// Initialize WebSocket connection for real-time updates
function initWebSocket() {
if (!wsEnabled) return;
// Check if rfMapperWS is available (websocket.js loaded)
if (typeof rfMapperWS === 'undefined') {
console.log('[App] WebSocket module not loaded, using HTTP polling');
return;
}
const connected = rfMapperWS.connect();
if (!connected) {
console.log('[App] WebSocket not available, using HTTP polling');
return;
}
rfMapperWS.on('connected', () => {
wsConnected = true;
console.log('[App] WebSocket connected');
// Stop HTTP polling if running (WS will handle updates)
if (liveTrackingInterval) {
clearInterval(liveTrackingInterval);
liveTrackingInterval = null;
console.log('[App] Stopped HTTP polling (using WebSocket)');
}
});
rfMapperWS.on('disconnected', (data) => {
wsConnected = false;
console.log('[App] WebSocket disconnected:', data?.reason);
// Resume HTTP polling if live tracking is enabled
if (liveTrackingEnabled && !liveTrackingInterval) {
liveTrackingInterval = setInterval(performLiveBTScan, LIVE_TRACKING_INTERVAL_MS);
console.log('[App] Resumed HTTP polling');
}
});
rfMapperWS.on('scanUpdate', (data) => {
handleWebSocketScanUpdate(data);
});
}
// Handle scan updates received via WebSocket
function handleWebSocketScanUpdate(data) {
if (!liveTrackingEnabled) return;
console.log('[WS] Scan update:', data.type, data.devices?.length, 'devices');
// Handle Bluetooth scan results
if (data.type === 'bluetooth' && data.devices) {
const newBt = data.devices;
// Track which devices were detected in this scan
const detectedAddresses = new Set(newBt.map(d => d.address));
if (scanData) {
const existingBt = scanData.bluetooth_devices || [];
// Update existing devices with new RSSI, add new devices
newBt.forEach(newDev => {
const existing = existingBt.find(d => d.address === newDev.address);
const newDist = newDev.estimated_distance_m;
// Check for movement using statistical analysis
const moving = isDeviceMoving(newDev.address, newDist);
// Reset miss count - device was detected
deviceMissCount[newDev.address] = 0;
if (existing) {
// Update RSSI and estimated distance, preserve custom values
existing.rssi = newDev.rssi;
existing.estimated_distance_m = newDev.estimated_distance_m;
existing.signal_quality = newDev.signal_quality;
existing.is_moving = moving;
existing.miss_count = 0;
// Preserve floor and custom_distance_m if set
} else {
// New device, add it
newDev.is_moving = moving;
existingBt.push(newDev);
}
});
// Increment miss count for devices not detected in this scan
existingBt.forEach(dev => {
if (!detectedAddresses.has(dev.address)) {
deviceMissCount[dev.address] = (deviceMissCount[dev.address] || 0) + 1;
dev.miss_count = deviceMissCount[dev.address];
}
});
// Filter out devices that have been missed too many times
const filteredBt = existingBt.filter(dev => {
const missCount = deviceMissCount[dev.address] || 0;
if (missCount >= MAX_MISSED_SCANS) {
// Clean up tracking data for removed device
delete deviceMissCount[dev.address];
delete deviceDistanceHistory[dev.address];
// Clear trail if showing
if (deviceTrails[dev.address]) {
clearDeviceTrail(dev.address);
}
console.log(`[WS] Removed ${dev.name} (missed ${missCount} scans)`);
return false;
}
return true;
});
scanData.bluetooth_devices = filteredBt;
} else {
// No existing scan data, use BT-only data
newBt.forEach(dev => {
// Initialize history with first sample, not moving yet
isDeviceMoving(dev.address, dev.estimated_distance_m);
dev.is_moving = false;
deviceMissCount[dev.address] = 0;
});
scanData = {
wifi_networks: [],
bluetooth_devices: newBt,
timestamp: data.timestamp
};
}
// Update visualizations
const status = document.getElementById('scan-status');
if (status) {
const movingCount = scanData.bluetooth_devices.filter(d => d.is_moving).length;
const wsIndicator = wsConnected ? '[WS]' : '';
status.textContent = `Live${wsIndicator}: ${scanData.bluetooth_devices.length} BT (${movingCount} moving) @ ${new Date().toLocaleTimeString()}`;
}
// Update BT count
document.getElementById('bt-count').textContent = scanData.bluetooth_devices.length;
document.getElementById('bt-list-count').textContent = scanData.bluetooth_devices.length;
// Refresh views
drawRadar();
update3DMarkers();
updateMapMarkers();
}
}
// Toggle filter
function toggleFilter(type) {
filters[type] = !filters[type];
@@ -1925,17 +2078,23 @@ function toggleLiveTracking() {
function startLiveTracking() {
if (liveTrackingInterval) {
clearInterval(liveTrackingInterval);
liveTrackingInterval = null;
}
liveTrackingEnabled = true;
updateLiveTrackingUI();
console.log('Live BT tracking started');
// Do initial scan
performLiveBTScan();
// Set up interval
liveTrackingInterval = setInterval(performLiveBTScan, LIVE_TRACKING_INTERVAL_MS);
if (wsConnected) {
// WebSocket mode - updates come automatically via 'scanUpdate' events
// Still need to trigger initial scan
performLiveBTScan();
console.log('[Live] Started (WebSocket mode)');
} else {
// HTTP polling fallback
liveTrackingInterval = setInterval(performLiveBTScan, LIVE_TRACKING_INTERVAL_MS);
performLiveBTScan();
console.log('[Live] Started (HTTP polling mode)');
}
}
// Stop live BT tracking

7
src/rf_mapper/web/static/js/vendor/socket.io.min.js generated vendored Normal file

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,101 @@
/**
* RF Mapper WebSocket client with automatic reconnection and HTTP fallback
*/
class RFMapperWS {
constructor() {
this.socket = null;
this.connected = false;
this.reconnectAttempts = 0;
this.maxReconnectAttempts = 5;
this.listeners = {
scanUpdate: [],
connected: [],
disconnected: []
};
}
connect() {
// Check if socket.io is loaded
if (typeof io === 'undefined') {
console.warn('[WS] socket.io not loaded, using HTTP polling');
return false;
}
try {
this.socket = io('/ws/scan', {
transports: ['websocket', 'polling'],
reconnection: true,
reconnectionDelay: 1000,
reconnectionDelayMax: 5000,
reconnectionAttempts: this.maxReconnectAttempts
});
this.socket.on('connect', () => {
console.log('[WS] Connected');
this.connected = true;
this.reconnectAttempts = 0;
this._emit('connected');
});
this.socket.on('disconnect', (reason) => {
console.log('[WS] Disconnected:', reason);
this.connected = false;
this._emit('disconnected', { reason });
});
this.socket.on('scan_update', (data) => {
this._emit('scanUpdate', data);
});
this.socket.on('connect_error', (error) => {
console.warn('[WS] Connection error:', error.message);
this.reconnectAttempts++;
if (this.reconnectAttempts >= this.maxReconnectAttempts) {
console.log('[WS] Max reconnect attempts, falling back to HTTP');
this.connected = false;
this._emit('disconnected', { reason: 'max_reconnect' });
}
});
return true;
} catch (e) {
console.error('[WS] Failed to initialize:', e);
return false;
}
}
disconnect() {
if (this.socket) {
this.socket.disconnect();
this.socket = null;
}
this.connected = false;
}
subscribeFloor(floor) {
if (this.socket?.connected) {
this.socket.emit('subscribe_floor', { floor });
}
}
on(event, callback) {
if (this.listeners[event]) {
this.listeners[event].push(callback);
}
}
off(event, callback) {
if (this.listeners[event]) {
this.listeners[event] = this.listeners[event].filter(cb => cb !== callback);
}
}
_emit(event, data) {
if (this.listeners[event]) {
this.listeners[event].forEach(cb => cb(data));
}
}
}
// Global instance
const rfMapperWS = new RFMapperWS();

View File

@@ -166,5 +166,10 @@
{% endblock %}
{% block extra_js %}
<!-- Socket.IO client -->
<script src="{{ url_for('static', filename='js/vendor/socket.io.min.js') }}"></script>
<!-- WebSocket client module -->
<script src="{{ url_for('static', filename='js/websocket.js') }}"></script>
<!-- Main application -->
<script src="{{ url_for('static', filename='js/app.js') }}"></script>
{% endblock %}