docs: update project docs for backoff, stale expiry, pool metrics
Add failure backoff and stale expiry sections to USAGE. Document pool= field in metrics output. Update ROADMAP, TASKS, TODO with completed items and remaining suggestions. Add metrics example to CHEATSHEET.
This commit is contained in:
@@ -54,3 +54,5 @@ All other functionality uses Python stdlib (`asyncio`, `socket`, `struct`).
|
|||||||
- **Config split** -- tracked example template, gitignored live config with real addresses
|
- **Config split** -- tracked example template, gitignored live config with real addresses
|
||||||
- **Proxy pool** -- multi-source (API + file), health-tested, persistent, auto-cleaned
|
- **Proxy pool** -- multi-source (API + file), health-tested, persistent, auto-cleaned
|
||||||
- **Weighted selection** -- recently-tested proxies preferred via recency decay weight
|
- **Weighted selection** -- recently-tested proxies preferred via recency decay weight
|
||||||
|
- **Failure backoff** -- connection failures penalize proxy weight for 60s, avoids retry waste
|
||||||
|
- **Stale expiry** -- proxies dropped from sources evicted after 3 refresh cycles if not alive
|
||||||
|
|||||||
@@ -12,8 +12,9 @@ through configurable chains of SOCKS4, SOCKS5, and HTTP CONNECT proxies.
|
|||||||
- DNS leak prevention (domain names forwarded to proxies, never resolved locally)
|
- DNS leak prevention (domain names forwarded to proxies, never resolved locally)
|
||||||
- Tor integration (Tor is just another SOCKS5 hop)
|
- Tor integration (Tor is just another SOCKS5 hop)
|
||||||
- Managed proxy pool: multiple sources (API + file), health-tested, weighted selection
|
- Managed proxy pool: multiple sources (API + file), health-tested, weighted selection
|
||||||
|
- Per-proxy failure backoff (60s cooldown) and stale proxy expiry
|
||||||
- Connection retry with proxy rotation (configurable attempts)
|
- Connection retry with proxy rotation (configurable attempts)
|
||||||
- Connection metrics (logged periodically and on shutdown)
|
- Connection metrics with pool stats (logged periodically and on shutdown)
|
||||||
- Container-ready (Alpine-based, podman/docker)
|
- Container-ready (Alpine-based, podman/docker)
|
||||||
- Graceful shutdown (SIGTERM/SIGINT)
|
- Graceful shutdown (SIGTERM/SIGINT)
|
||||||
- Pure Python, asyncio-based, minimal dependencies
|
- Pure Python, asyncio-based, minimal dependencies
|
||||||
|
|||||||
@@ -12,12 +12,15 @@
|
|||||||
- [x] cProfile support
|
- [x] cProfile support
|
||||||
- [x] Dynamic proxy source API integration
|
- [x] Dynamic proxy source API integration
|
||||||
- [x] Weighted proxy selection (recency-based)
|
- [x] Weighted proxy selection (recency-based)
|
||||||
|
- [x] Per-proxy backoff (connection failure cooldown)
|
||||||
|
- [x] Stale proxy expiry (last_seen TTL)
|
||||||
|
- [x] Pool stats in periodic metrics log
|
||||||
|
|
||||||
## v0.2.0
|
## v0.2.0
|
||||||
|
|
||||||
- [ ] SOCKS5 server authentication (username/password)
|
- [ ] SOCKS5 server authentication (username/password)
|
||||||
- [ ] Tor control port integration (circuit renewal via NEWNYM)
|
- [ ] Tor control port integration (circuit renewal via NEWNYM)
|
||||||
- [ ] Connection retry with configurable backoff
|
- [ ] Parallel health tests at startup (fast warm start)
|
||||||
- [ ] Metrics (connections/sec, bytes relayed, hop latency)
|
- [ ] Metrics (connections/sec, bytes relayed, hop latency)
|
||||||
|
|
||||||
## v0.3.0
|
## v0.3.0
|
||||||
|
|||||||
3
TASKS.md
3
TASKS.md
@@ -20,6 +20,9 @@
|
|||||||
- [x] Connection metrics (periodic + shutdown logging)
|
- [x] Connection metrics (periodic + shutdown logging)
|
||||||
- [x] Managed proxy pool (multi-source, health-tested, persistent)
|
- [x] Managed proxy pool (multi-source, health-tested, persistent)
|
||||||
- [x] Weighted proxy selection (prefer recently-tested proxies)
|
- [x] Weighted proxy selection (prefer recently-tested proxies)
|
||||||
|
- [x] Per-proxy backoff (60s cooldown after connection failure)
|
||||||
|
- [x] Stale proxy expiry (evict dead proxies not seen for 3 refresh cycles)
|
||||||
|
- [x] Pool stats in periodic metrics log (`pool=alive/total`)
|
||||||
|
|
||||||
## Next
|
## Next
|
||||||
- [ ] Integration tests with mock proxy server
|
- [ ] Integration tests with mock proxy server
|
||||||
|
|||||||
3
TODO.md
3
TODO.md
@@ -7,6 +7,9 @@
|
|||||||
- Per-destination chain rules (bypass chain for local addresses)
|
- Per-destination chain rules (bypass chain for local addresses)
|
||||||
- Hot config reload on SIGHUP
|
- Hot config reload on SIGHUP
|
||||||
- Systemd socket activation
|
- Systemd socket activation
|
||||||
|
- Parallel health tests at startup (fast warm start)
|
||||||
|
- Report dead proxies back to source API
|
||||||
|
- Chain health check (test static chain before pool tests)
|
||||||
|
|
||||||
## Performance
|
## Performance
|
||||||
|
|
||||||
|
|||||||
@@ -87,6 +87,12 @@ curl --max-time 30 -x socks5h://127.0.0.1:1080 https://example.com
|
|||||||
python -m pstats s5p.prof # interactive stats viewer
|
python -m pstats s5p.prof # interactive stats viewer
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Metrics Log
|
||||||
|
|
||||||
|
```
|
||||||
|
metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s pool=42/65
|
||||||
|
```
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
| Symptom | Check |
|
| Symptom | Check |
|
||||||
@@ -97,3 +103,4 @@ python -m pstats s5p.prof # interactive stats viewer
|
|||||||
| Auth failed | Verify credentials in proxy URL |
|
| Auth failed | Verify credentials in proxy URL |
|
||||||
| Port in use | `fuser -k 1080/tcp` to free the port |
|
| Port in use | `fuser -k 1080/tcp` to free the port |
|
||||||
| Container slow stop | Rebuild image after SIGTERM fix |
|
| Container slow stop | Rebuild image after SIGTERM fix |
|
||||||
|
| Proxy keeps failing | Backoff penalizes for 60s; check `pool=` in metrics |
|
||||||
|
|||||||
@@ -148,6 +148,19 @@ successful health test. This favors freshly-verified proxies over stale ones:
|
|||||||
| 30 min | ~0.1 |
|
| 30 min | ~0.1 |
|
||||||
| Never tested | 0.01 |
|
| Never tested | 0.01 |
|
||||||
|
|
||||||
|
### Failure backoff
|
||||||
|
|
||||||
|
When a proxy fails during an actual connection attempt (not just a health
|
||||||
|
test), its weight is penalized for 60 seconds. The penalty ramps linearly
|
||||||
|
from floor (0.01) back to normal over that window. This prevents retries
|
||||||
|
from repeatedly selecting a proxy that just failed.
|
||||||
|
|
||||||
|
### Stale proxy expiry
|
||||||
|
|
||||||
|
Proxies not returned by any source for 3 consecutive refresh cycles and
|
||||||
|
not currently alive are automatically evicted. This cleans up proxies
|
||||||
|
removed upstream faster than waiting for `max_fails` health test failures.
|
||||||
|
|
||||||
### Persistence
|
### Persistence
|
||||||
|
|
||||||
Pool state is saved to `state_file` (default: `~/.cache/s5p/pool.json`) after
|
Pool state is saved to `state_file` (default: `~/.cache/s5p/pool.json`) after
|
||||||
@@ -188,7 +201,7 @@ s5p tracks connection metrics and logs a summary every 60 seconds and on
|
|||||||
shutdown:
|
shutdown:
|
||||||
|
|
||||||
```
|
```
|
||||||
metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s
|
metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s pool=42/65
|
||||||
```
|
```
|
||||||
|
|
||||||
| Counter | Meaning |
|
| Counter | Meaning |
|
||||||
@@ -201,6 +214,7 @@ metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s
|
|||||||
| `in` | Bytes client -> remote |
|
| `in` | Bytes client -> remote |
|
||||||
| `out` | Bytes remote -> client |
|
| `out` | Bytes remote -> client |
|
||||||
| `up` | Server uptime |
|
| `up` | Server uptime |
|
||||||
|
| `pool` | Alive/total proxies (only when pool is active) |
|
||||||
|
|
||||||
## Profiling
|
## Profiling
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user