docs: mark tor connection pooling complete
This commit is contained in:
47
TODO.md
47
TODO.md
@@ -124,46 +124,15 @@ and report() methods. Integrated into main loop with configurable stats_interval
|
||||
|
||||
## Medium Term (Next Quarter)
|
||||
|
||||
### [ ] 11. Tor Connection Pooling
|
||||
### [x] 11. Tor Connection Pooling
|
||||
|
||||
**Problem:** Each proxy test creates a new Tor connection. Tor circuit establishment
|
||||
is slow (~2-3 seconds).
|
||||
|
||||
**Implementation:**
|
||||
```python
|
||||
# new file: connection_pool.py
|
||||
class TorConnectionPool:
|
||||
"""Pool of reusable Tor SOCKS connections."""
|
||||
|
||||
def __init__(self, tor_hosts, pool_size=10):
|
||||
self.tor_hosts = tor_hosts
|
||||
self.pool_size = pool_size
|
||||
self.connections = Queue.Queue(pool_size)
|
||||
self.lock = threading.Lock()
|
||||
|
||||
def get(self, timeout=5):
|
||||
"""Get a Tor connection from pool, or create new."""
|
||||
try:
|
||||
return self.connections.get(timeout=0.1)
|
||||
except Queue.Empty:
|
||||
return self._create_connection()
|
||||
|
||||
def release(self, conn):
|
||||
"""Return connection to pool."""
|
||||
try:
|
||||
self.connections.put_nowait(conn)
|
||||
except Queue.Full:
|
||||
conn.close()
|
||||
|
||||
def _create_connection(self):
|
||||
"""Create new Tor SOCKS connection."""
|
||||
host = random.choice(self.tor_hosts)
|
||||
# ... establish connection
|
||||
```
|
||||
|
||||
**Files:** new connection_pool.py, proxywatchd.py
|
||||
**Effort:** High
|
||||
**Risk:** Medium
|
||||
**Completed.** Added connection pooling with worker-Tor affinity and health monitoring.
|
||||
- connection_pool.py: TorHostState class tracks per-host health, latency, backoff
|
||||
- connection_pool.py: TorConnectionPool with worker affinity, warmup, statistics
|
||||
- proxywatchd.py: Workers get consistent Tor host assignment for circuit reuse
|
||||
- Success/failure tracking with exponential backoff (5s, 10s, 20s, 40s, max 60s)
|
||||
- Latency tracking with rolling averages
|
||||
- Pool status reported alongside periodic stats
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user