docs: update project docs for backoff, stale expiry, pool metrics

Add failure backoff and stale expiry sections to USAGE. Document pool=
field in metrics output. Update ROADMAP, TASKS, TODO with completed
items and remaining suggestions. Add metrics example to CHEATSHEET.
This commit is contained in:
user
2026-02-15 15:56:06 +01:00
parent 8aa384a80b
commit eddcc5f615
7 changed files with 36 additions and 3 deletions

View File

@@ -54,3 +54,5 @@ All other functionality uses Python stdlib (`asyncio`, `socket`, `struct`).
- **Config split** -- tracked example template, gitignored live config with real addresses
- **Proxy pool** -- multi-source (API + file), health-tested, persistent, auto-cleaned
- **Weighted selection** -- recently-tested proxies preferred via recency decay weight
- **Failure backoff** -- connection failures penalize proxy weight for 60s, avoids retry waste
- **Stale expiry** -- proxies dropped from sources evicted after 3 refresh cycles if not alive

View File

@@ -12,8 +12,9 @@ through configurable chains of SOCKS4, SOCKS5, and HTTP CONNECT proxies.
- DNS leak prevention (domain names forwarded to proxies, never resolved locally)
- Tor integration (Tor is just another SOCKS5 hop)
- Managed proxy pool: multiple sources (API + file), health-tested, weighted selection
- Per-proxy failure backoff (60s cooldown) and stale proxy expiry
- Connection retry with proxy rotation (configurable attempts)
- Connection metrics (logged periodically and on shutdown)
- Connection metrics with pool stats (logged periodically and on shutdown)
- Container-ready (Alpine-based, podman/docker)
- Graceful shutdown (SIGTERM/SIGINT)
- Pure Python, asyncio-based, minimal dependencies

View File

@@ -12,12 +12,15 @@
- [x] cProfile support
- [x] Dynamic proxy source API integration
- [x] Weighted proxy selection (recency-based)
- [x] Per-proxy backoff (connection failure cooldown)
- [x] Stale proxy expiry (last_seen TTL)
- [x] Pool stats in periodic metrics log
## v0.2.0
- [ ] SOCKS5 server authentication (username/password)
- [ ] Tor control port integration (circuit renewal via NEWNYM)
- [ ] Connection retry with configurable backoff
- [ ] Parallel health tests at startup (fast warm start)
- [ ] Metrics (connections/sec, bytes relayed, hop latency)
## v0.3.0

View File

@@ -20,6 +20,9 @@
- [x] Connection metrics (periodic + shutdown logging)
- [x] Managed proxy pool (multi-source, health-tested, persistent)
- [x] Weighted proxy selection (prefer recently-tested proxies)
- [x] Per-proxy backoff (60s cooldown after connection failure)
- [x] Stale proxy expiry (evict dead proxies not seen for 3 refresh cycles)
- [x] Pool stats in periodic metrics log (`pool=alive/total`)
## Next
- [ ] Integration tests with mock proxy server

View File

@@ -7,6 +7,9 @@
- Per-destination chain rules (bypass chain for local addresses)
- Hot config reload on SIGHUP
- Systemd socket activation
- Parallel health tests at startup (fast warm start)
- Report dead proxies back to source API
- Chain health check (test static chain before pool tests)
## Performance

View File

@@ -87,6 +87,12 @@ curl --max-time 30 -x socks5h://127.0.0.1:1080 https://example.com
python -m pstats s5p.prof # interactive stats viewer
```
## Metrics Log
```
metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s pool=42/65
```
## Troubleshooting
| Symptom | Check |
@@ -97,3 +103,4 @@ python -m pstats s5p.prof # interactive stats viewer
| Auth failed | Verify credentials in proxy URL |
| Port in use | `fuser -k 1080/tcp` to free the port |
| Container slow stop | Rebuild image after SIGTERM fix |
| Proxy keeps failing | Backoff penalizes for 60s; check `pool=` in metrics |

View File

@@ -148,6 +148,19 @@ successful health test. This favors freshly-verified proxies over stale ones:
| 30 min | ~0.1 |
| Never tested | 0.01 |
### Failure backoff
When a proxy fails during an actual connection attempt (not just a health
test), its weight is penalized for 60 seconds. The penalty ramps linearly
from floor (0.01) back to normal over that window. This prevents retries
from repeatedly selecting a proxy that just failed.
### Stale proxy expiry
Proxies not returned by any source for 3 consecutive refresh cycles and
not currently alive are automatically evicted. This cleans up proxies
removed upstream faster than waiting for `max_fails` health test failures.
### Persistence
Pool state is saved to `state_file` (default: `~/.cache/s5p/pool.json`) after
@@ -188,7 +201,7 @@ s5p tracks connection metrics and logs a summary every 60 seconds and on
shutdown:
```
metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s
metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s pool=42/65
```
| Counter | Meaning |
@@ -201,6 +214,7 @@ metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s
| `in` | Bytes client -> remote |
| `out` | Bytes remote -> client |
| `up` | Server uptime |
| `pool` | Alive/total proxies (only when pool is active) |
## Profiling