diff --git a/TODO.md b/TODO.md index cb372f4..2108387 100644 --- a/TODO.md +++ b/TODO.md @@ -144,48 +144,20 @@ and report() methods. Integrated into main loop with configurable stats_interval --- -### [ ] 15. Unit Test Infrastructure +### [x] 15. Unit Test Infrastructure -**Problem:** No automated tests. Changes can break existing functionality silently. +**Completed.** Added pytest-based test suite with comprehensive coverage. +- tests/conftest.py: Shared fixtures (temp_db, proxy_db, sample_proxies, etc.) +- tests/test_dbs.py: 40 tests for database operations (CDN filtering, latency, stats) +- tests/test_fetch.py: 60 tests for proxy validation (skipped in Python 3) +- tests/test_misc.py: 39 tests for utilities (timestamp, log levels, SSL errors) +- tests/mock_network.py: Network mocking infrastructure -**Implementation:** ``` -tests/ -├── __init__.py -├── test_proxy_utils.py # Test IP validation, cleansing -├── test_extract.py # Test proxy/URL extraction -├── test_database.py # Test DB operations with temp DB -└── mock_network.py # Mock rocksock for offline testing +Test Results: 79 passed, 60 skipped (Python 2 only) +Run with: python3 -m pytest tests/ -v ``` -```python -# tests/test_proxy_utils.py -import unittest -import sys -sys.path.insert(0, '..') -import fetch - -class TestProxyValidation(unittest.TestCase): - def test_valid_proxy(self): - self.assertTrue(fetch.is_usable_proxy('8.8.8.8:8080')) - - def test_private_ip_rejected(self): - self.assertFalse(fetch.is_usable_proxy('192.168.1.1:8080')) - self.assertFalse(fetch.is_usable_proxy('10.0.0.1:8080')) - self.assertFalse(fetch.is_usable_proxy('172.16.0.1:8080')) - - def test_invalid_port_rejected(self): - self.assertFalse(fetch.is_usable_proxy('8.8.8.8:0')) - self.assertFalse(fetch.is_usable_proxy('8.8.8.8:99999')) - -if __name__ == '__main__': - unittest.main() -``` - -**Files:** tests/ directory -**Effort:** High (initial), Low (ongoing) -**Risk:** Low - --- ## Long Term (Future) @@ -261,6 +233,31 @@ if __name__ == '__main__': --- +### [x] 23. Batch API Endpoint + +**Completed.** Added `/api/dashboard` batch endpoint combining stats, workers, and countries. + +**Implementation:** +- `httpd.py`: New `/api/dashboard` endpoint returns combined data in single response +- `httpd.py`: Refactored `/api/workers` to use `_get_workers_data()` helper method +- `dashboard.js`: Updated `fetchStats()` to use batch endpoint instead of multiple calls + +**Response Structure:** +```json +{ + "stats": { /* same as /api/stats */ }, + "workers": { /* same as /api/workers */ }, + "countries": { /* same as /api/countries */ } +} +``` + +**Benefit:** +- Reduces dashboard polling from 2 HTTP requests to 1 per poll cycle +- Lower RTT impact over SSH tunnels and high-latency connections +- Single database connection serves all data + +--- + ## Profiling-Based Performance Optimizations **Baseline:** 30-minute profiling session, 25.6M function calls, 1842s runtime @@ -446,6 +443,8 @@ above target the remaining 31.3% of CPU-bound operations. - [ ] Configurable refresh interval via URL param or localStorage - [x] Pause polling when browser tab not visible (Page Visibility API) - [x] Skip chart rendering for inactive dashboard tabs (reduces CPU) +- [ ] Batch API endpoints - combine /api/stats, /api/workers, /api/countries into + single /api/dashboard call to reduce round-trips (helps SSH tunnel latency) ### [ ] Dashboard Feature Ideas