docs: document batch API endpoint
This commit is contained in:
73
TODO.md
73
TODO.md
@@ -144,48 +144,20 @@ and report() methods. Integrated into main loop with configurable stats_interval
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### [ ] 15. Unit Test Infrastructure
|
### [x] 15. Unit Test Infrastructure
|
||||||
|
|
||||||
**Problem:** No automated tests. Changes can break existing functionality silently.
|
**Completed.** Added pytest-based test suite with comprehensive coverage.
|
||||||
|
- tests/conftest.py: Shared fixtures (temp_db, proxy_db, sample_proxies, etc.)
|
||||||
|
- tests/test_dbs.py: 40 tests for database operations (CDN filtering, latency, stats)
|
||||||
|
- tests/test_fetch.py: 60 tests for proxy validation (skipped in Python 3)
|
||||||
|
- tests/test_misc.py: 39 tests for utilities (timestamp, log levels, SSL errors)
|
||||||
|
- tests/mock_network.py: Network mocking infrastructure
|
||||||
|
|
||||||
**Implementation:**
|
|
||||||
```
|
```
|
||||||
tests/
|
Test Results: 79 passed, 60 skipped (Python 2 only)
|
||||||
├── __init__.py
|
Run with: python3 -m pytest tests/ -v
|
||||||
├── test_proxy_utils.py # Test IP validation, cleansing
|
|
||||||
├── test_extract.py # Test proxy/URL extraction
|
|
||||||
├── test_database.py # Test DB operations with temp DB
|
|
||||||
└── mock_network.py # Mock rocksock for offline testing
|
|
||||||
```
|
```
|
||||||
|
|
||||||
```python
|
|
||||||
# tests/test_proxy_utils.py
|
|
||||||
import unittest
|
|
||||||
import sys
|
|
||||||
sys.path.insert(0, '..')
|
|
||||||
import fetch
|
|
||||||
|
|
||||||
class TestProxyValidation(unittest.TestCase):
|
|
||||||
def test_valid_proxy(self):
|
|
||||||
self.assertTrue(fetch.is_usable_proxy('8.8.8.8:8080'))
|
|
||||||
|
|
||||||
def test_private_ip_rejected(self):
|
|
||||||
self.assertFalse(fetch.is_usable_proxy('192.168.1.1:8080'))
|
|
||||||
self.assertFalse(fetch.is_usable_proxy('10.0.0.1:8080'))
|
|
||||||
self.assertFalse(fetch.is_usable_proxy('172.16.0.1:8080'))
|
|
||||||
|
|
||||||
def test_invalid_port_rejected(self):
|
|
||||||
self.assertFalse(fetch.is_usable_proxy('8.8.8.8:0'))
|
|
||||||
self.assertFalse(fetch.is_usable_proxy('8.8.8.8:99999'))
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
unittest.main()
|
|
||||||
```
|
|
||||||
|
|
||||||
**Files:** tests/ directory
|
|
||||||
**Effort:** High (initial), Low (ongoing)
|
|
||||||
**Risk:** Low
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Long Term (Future)
|
## Long Term (Future)
|
||||||
@@ -261,6 +233,31 @@ if __name__ == '__main__':
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
### [x] 23. Batch API Endpoint
|
||||||
|
|
||||||
|
**Completed.** Added `/api/dashboard` batch endpoint combining stats, workers, and countries.
|
||||||
|
|
||||||
|
**Implementation:**
|
||||||
|
- `httpd.py`: New `/api/dashboard` endpoint returns combined data in single response
|
||||||
|
- `httpd.py`: Refactored `/api/workers` to use `_get_workers_data()` helper method
|
||||||
|
- `dashboard.js`: Updated `fetchStats()` to use batch endpoint instead of multiple calls
|
||||||
|
|
||||||
|
**Response Structure:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"stats": { /* same as /api/stats */ },
|
||||||
|
"workers": { /* same as /api/workers */ },
|
||||||
|
"countries": { /* same as /api/countries */ }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Benefit:**
|
||||||
|
- Reduces dashboard polling from 2 HTTP requests to 1 per poll cycle
|
||||||
|
- Lower RTT impact over SSH tunnels and high-latency connections
|
||||||
|
- Single database connection serves all data
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Profiling-Based Performance Optimizations
|
## Profiling-Based Performance Optimizations
|
||||||
|
|
||||||
**Baseline:** 30-minute profiling session, 25.6M function calls, 1842s runtime
|
**Baseline:** 30-minute profiling session, 25.6M function calls, 1842s runtime
|
||||||
@@ -446,6 +443,8 @@ above target the remaining 31.3% of CPU-bound operations.
|
|||||||
- [ ] Configurable refresh interval via URL param or localStorage
|
- [ ] Configurable refresh interval via URL param or localStorage
|
||||||
- [x] Pause polling when browser tab not visible (Page Visibility API)
|
- [x] Pause polling when browser tab not visible (Page Visibility API)
|
||||||
- [x] Skip chart rendering for inactive dashboard tabs (reduces CPU)
|
- [x] Skip chart rendering for inactive dashboard tabs (reduces CPU)
|
||||||
|
- [ ] Batch API endpoints - combine /api/stats, /api/workers, /api/countries into
|
||||||
|
single /api/dashboard call to reduce round-trips (helps SSH tunnel latency)
|
||||||
|
|
||||||
### [ ] Dashboard Feature Ideas
|
### [ ] Dashboard Feature Ideas
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user