diff --git a/PROJECT.md b/PROJECT.md index 8cbf2af..416423c 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -54,3 +54,5 @@ All other functionality uses Python stdlib (`asyncio`, `socket`, `struct`). - **Config split** -- tracked example template, gitignored live config with real addresses - **Proxy pool** -- multi-source (API + file), health-tested, persistent, auto-cleaned - **Weighted selection** -- recently-tested proxies preferred via recency decay weight +- **Failure backoff** -- connection failures penalize proxy weight for 60s, avoids retry waste +- **Stale expiry** -- proxies dropped from sources evicted after 3 refresh cycles if not alive diff --git a/README.md b/README.md index 3ed7912..1151c08 100644 --- a/README.md +++ b/README.md @@ -12,8 +12,9 @@ through configurable chains of SOCKS4, SOCKS5, and HTTP CONNECT proxies. - DNS leak prevention (domain names forwarded to proxies, never resolved locally) - Tor integration (Tor is just another SOCKS5 hop) - Managed proxy pool: multiple sources (API + file), health-tested, weighted selection +- Per-proxy failure backoff (60s cooldown) and stale proxy expiry - Connection retry with proxy rotation (configurable attempts) -- Connection metrics (logged periodically and on shutdown) +- Connection metrics with pool stats (logged periodically and on shutdown) - Container-ready (Alpine-based, podman/docker) - Graceful shutdown (SIGTERM/SIGINT) - Pure Python, asyncio-based, minimal dependencies diff --git a/ROADMAP.md b/ROADMAP.md index eb4b713..3ab4ab1 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -12,12 +12,15 @@ - [x] cProfile support - [x] Dynamic proxy source API integration - [x] Weighted proxy selection (recency-based) +- [x] Per-proxy backoff (connection failure cooldown) +- [x] Stale proxy expiry (last_seen TTL) +- [x] Pool stats in periodic metrics log ## v0.2.0 - [ ] SOCKS5 server authentication (username/password) - [ ] Tor control port integration (circuit renewal via NEWNYM) -- [ ] Connection retry with configurable backoff +- [ ] Parallel health tests at startup (fast warm start) - [ ] Metrics (connections/sec, bytes relayed, hop latency) ## v0.3.0 diff --git a/TASKS.md b/TASKS.md index a90c57d..a3a2fc9 100644 --- a/TASKS.md +++ b/TASKS.md @@ -20,6 +20,9 @@ - [x] Connection metrics (periodic + shutdown logging) - [x] Managed proxy pool (multi-source, health-tested, persistent) - [x] Weighted proxy selection (prefer recently-tested proxies) +- [x] Per-proxy backoff (60s cooldown after connection failure) +- [x] Stale proxy expiry (evict dead proxies not seen for 3 refresh cycles) +- [x] Pool stats in periodic metrics log (`pool=alive/total`) ## Next - [ ] Integration tests with mock proxy server diff --git a/TODO.md b/TODO.md index 1c0503f..d8c2a3d 100644 --- a/TODO.md +++ b/TODO.md @@ -7,6 +7,9 @@ - Per-destination chain rules (bypass chain for local addresses) - Hot config reload on SIGHUP - Systemd socket activation +- Parallel health tests at startup (fast warm start) +- Report dead proxies back to source API +- Chain health check (test static chain before pool tests) ## Performance diff --git a/docs/CHEATSHEET.md b/docs/CHEATSHEET.md index 5c440cd..afd4e00 100644 --- a/docs/CHEATSHEET.md +++ b/docs/CHEATSHEET.md @@ -87,6 +87,12 @@ curl --max-time 30 -x socks5h://127.0.0.1:1080 https://example.com python -m pstats s5p.prof # interactive stats viewer ``` +## Metrics Log + +``` +metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s pool=42/65 +``` + ## Troubleshooting | Symptom | Check | @@ -97,3 +103,4 @@ python -m pstats s5p.prof # interactive stats viewer | Auth failed | Verify credentials in proxy URL | | Port in use | `fuser -k 1080/tcp` to free the port | | Container slow stop | Rebuild image after SIGTERM fix | +| Proxy keeps failing | Backoff penalizes for 60s; check `pool=` in metrics | diff --git a/docs/USAGE.md b/docs/USAGE.md index dcf72a6..e2d86c9 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -148,6 +148,19 @@ successful health test. This favors freshly-verified proxies over stale ones: | 30 min | ~0.1 | | Never tested | 0.01 | +### Failure backoff + +When a proxy fails during an actual connection attempt (not just a health +test), its weight is penalized for 60 seconds. The penalty ramps linearly +from floor (0.01) back to normal over that window. This prevents retries +from repeatedly selecting a proxy that just failed. + +### Stale proxy expiry + +Proxies not returned by any source for 3 consecutive refresh cycles and +not currently alive are automatically evicted. This cleans up proxies +removed upstream faster than waiting for `max_fails` health test failures. + ### Persistence Pool state is saved to `state_file` (default: `~/.cache/s5p/pool.json`) after @@ -188,7 +201,7 @@ s5p tracks connection metrics and logs a summary every 60 seconds and on shutdown: ``` -metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s +metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s pool=42/65 ``` | Counter | Meaning | @@ -201,6 +214,7 @@ metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s | `in` | Bytes client -> remote | | `out` | Bytes remote -> client | | `up` | Server uptime | +| `pool` | Alive/total proxies (only when pool is active) | ## Profiling