docs: update roadmap with completed items
This commit is contained in:
19
ROADMAP.md
19
ROADMAP.md
@@ -179,14 +179,14 @@ PPF (Proxy Fetcher) is a Python 2 proxy scraping and validation framework design
|
||||
│ │ │
|
||||
│ [x] Unify _known_proxies │ [x] Connection pooling │
|
||||
│ [x] Graceful DB errors │ [x] Dynamic thread scaling │
|
||||
│ [x] Batch inserts │ [ ] Unit test infrastructure │
|
||||
│ [x] Batch inserts │ [x] Unit test infrastructure │
|
||||
│ [x] WAL mode for SQLite │ [x] Latency tracking │
|
||||
│ │ │
|
||||
├──────────────────────────┼──────────────────────────────────────────────────┤
|
||||
│ LOW IMPACT / LOW EFFORT │ LOW IMPACT / HIGH EFFORT │
|
||||
│ │ │
|
||||
│ [x] Standardize logging │ [x] Geographic validation │
|
||||
│ [x] Config validation │ [x] Additional scrapers │
|
||||
│ [x] Config validation │ [x] Additional scrapers (Bing, Yahoo, Mojeek) │
|
||||
│ [x] Export functionality │ [ ] API sources │
|
||||
│ [x] Status output │ [ ] Protocol fingerprinting │
|
||||
│ │ │
|
||||
@@ -342,6 +342,7 @@ PPF (Proxy Fetcher) is a Python 2 proxy scraping and validation framework design
|
||||
- [x] Tab-aware chart rendering - skip expensive renders for hidden tabs
|
||||
- [x] Visibility API - pause polling when browser tab hidden
|
||||
- [x] Dark/muted-dark/light theme cycling
|
||||
- [x] Stats export endpoint (/api/stats/export?format=json|csv)
|
||||
|
||||
### Proxy Validation Cache (Done)
|
||||
- [x] LRU cache for is_usable_proxy() using OrderedDict
|
||||
@@ -354,6 +355,20 @@ PPF (Proxy Fetcher) is a Python 2 proxy scraping and validation framework design
|
||||
- [x] Removed deprecated `_prep_db()` and `_close_db()` methods
|
||||
- [x] `fetch_rows()` now accepts db parameter for cleaner dependency injection
|
||||
|
||||
### Additional Search Engines (Done)
|
||||
- [x] Bing and Yahoo engine implementations in scraper.py
|
||||
- [x] Engine rotation for rate limit avoidance
|
||||
- [x] engines.py module with SearchEngine base class
|
||||
|
||||
### Worker Health Improvements (Done)
|
||||
- [x] Tor connectivity check before workers claim work
|
||||
- [x] Fixed interval Tor check (30s) instead of exponential backoff
|
||||
- [x] Graceful handling when Tor unavailable
|
||||
|
||||
### Memory Optimization (Done)
|
||||
- [x] `__slots__` on ProxyTestState (27 attrs) and TargetTestJob (4 attrs)
|
||||
- [x] Reduced per-object memory overhead for hot path objects
|
||||
|
||||
---
|
||||
|
||||
## Technical Debt
|
||||
|
||||
Reference in New Issue
Block a user