docs: add proxy pool documentation
Update all docs for managed proxy pool: README, USAGE, CHEATSHEET, PROJECT, TASKS, and example config. Document multi-source config, proxy file format, health testing, persistence, and legacy compat. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -41,18 +41,25 @@ cp config/example.yaml config/s5p.yaml
|
||||
```yaml
|
||||
listen: 127.0.0.1:1080
|
||||
timeout: 10
|
||||
retries: 3 # max attempts per connection (proxy_source only)
|
||||
retries: 3
|
||||
log_level: info
|
||||
|
||||
chain:
|
||||
- socks5://127.0.0.1:9050
|
||||
|
||||
proxy_source:
|
||||
url: http://10.200.1.250:8081/proxies
|
||||
proto: socks5 # optional: filter by protocol
|
||||
country: US # optional: filter by country
|
||||
limit: 1000 # max proxies to fetch
|
||||
refresh: 300 # cache refresh interval (seconds)
|
||||
proxy_pool:
|
||||
sources:
|
||||
- url: http://10.200.1.250:8081/proxies
|
||||
proto: socks5
|
||||
limit: 1000
|
||||
- file: /etc/s5p/proxies.txt
|
||||
refresh: 300
|
||||
test_interval: 120
|
||||
test_url: http://httpbin.org/ip
|
||||
test_timeout: 15
|
||||
test_concurrency: 5
|
||||
max_fails: 3
|
||||
state_file: "" # empty = ~/.cache/s5p/pool.json
|
||||
```
|
||||
|
||||
## Proxy URL Format
|
||||
@@ -79,32 +86,76 @@ make down # stop and remove container
|
||||
Source (`./src`) and config (`./config/s5p.yaml`) are mounted read-only
|
||||
into the container. Edit locally, restart to pick up changes.
|
||||
|
||||
## Proxy Source
|
||||
## Proxy Pool
|
||||
|
||||
Appends a random proxy from an HTTP API after the static chain on each
|
||||
connection. Proxies are cached and refreshed at a configurable interval.
|
||||
Managed proxy pool with multiple sources, health testing, and persistence.
|
||||
Appends a random alive proxy after the static chain on each connection.
|
||||
|
||||
```yaml
|
||||
proxy_source:
|
||||
url: http://10.200.1.250:8081/proxies
|
||||
proto: socks5 # optional: only fetch this protocol
|
||||
country: US # optional: only fetch this country
|
||||
limit: 1000 # max proxies to fetch from API
|
||||
refresh: 300 # re-fetch every 300 seconds
|
||||
proxy_pool:
|
||||
sources:
|
||||
- url: http://10.200.1.250:8081/proxies
|
||||
proto: socks5 # optional: filter by protocol
|
||||
country: US # optional: filter by country
|
||||
limit: 1000 # max proxies to fetch from API
|
||||
- file: /etc/s5p/proxies.txt # text file, one proxy URL per line
|
||||
refresh: 300 # re-fetch sources every 300 seconds
|
||||
test_interval: 120 # health test cycle every 120 seconds
|
||||
test_url: http://httpbin.org/ip # URL for health checks
|
||||
test_timeout: 15 # per-test timeout (seconds)
|
||||
test_concurrency: 5 # parallel health tests
|
||||
max_fails: 3 # evict after N consecutive failures
|
||||
state_file: "" # empty = ~/.cache/s5p/pool.json
|
||||
```
|
||||
|
||||
CLI shorthand (uses defaults for limit/refresh):
|
||||
### Sources
|
||||
|
||||
| Type | Config key | Description |
|
||||
|------|-----------|-------------|
|
||||
| HTTP API | `url` | JSON: `{"proxies": [{"proto": "socks5", "proxy": "host:port"}, ...]}` |
|
||||
| Text file | `file` | One proxy URL per line, `#` comments, blank lines ignored |
|
||||
|
||||
### Proxy file format
|
||||
|
||||
```
|
||||
# Exit proxies
|
||||
socks5://1.2.3.4:1080
|
||||
socks5://user:pass@5.6.7.8:1080
|
||||
http://proxy.example.com:8080
|
||||
```
|
||||
|
||||
### Health testing
|
||||
|
||||
Each cycle tests all proxies through the full chain (static chain + proxy)
|
||||
by sending an HTTP GET to `test_url`. Proxies are marked alive on `200` response.
|
||||
After `max_fails` consecutive failures, a proxy is evicted.
|
||||
|
||||
Mass-failure guard: if >90% of tests fail in one cycle, eviction is skipped
|
||||
(likely the static chain is broken, not the proxies).
|
||||
|
||||
### Persistence
|
||||
|
||||
Pool state is saved to `state_file` (default: `~/.cache/s5p/pool.json`) after
|
||||
each refresh/health cycle and on shutdown. On startup, previously-alive proxies
|
||||
are loaded for fast warm starts.
|
||||
|
||||
### CLI shorthand
|
||||
|
||||
```bash
|
||||
s5p -C socks5://127.0.0.1:9050 -S http://10.200.1.250:8081/proxies
|
||||
```
|
||||
|
||||
The API must return JSON: `{"proxies": [{"proto": "socks5", "proxy": "host:port"}, ...]}`.
|
||||
Entries with `null` proto are skipped.
|
||||
The `-S` flag creates a pool with a single API source (uses defaults for all
|
||||
other pool settings).
|
||||
|
||||
### Legacy config
|
||||
|
||||
The old `proxy_source` key is still supported and auto-converts to `proxy_pool`
|
||||
with a single API source. `proxy_pool` takes precedence if both are present.
|
||||
|
||||
## Connection Retry
|
||||
|
||||
When `proxy_source` is active, s5p retries failed connections with a different
|
||||
When a proxy pool is active, s5p retries failed connections with a different
|
||||
random proxy. Controlled by the `retries` setting (default: 3). Static-only
|
||||
chains do not retry (retrying the same chain is pointless).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user