Update all docs for managed proxy pool: README, USAGE, CHEATSHEET, PROJECT, TASKS, and example config. Document multi-source config, proxy file format, health testing, persistence, and legacy compat. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
228 lines
5.6 KiB
Markdown
228 lines
5.6 KiB
Markdown
# s5p -- Usage
|
|
|
|
## Basic Usage
|
|
|
|
```bash
|
|
# Direct proxy (no chain, just a SOCKS5 server)
|
|
s5p
|
|
|
|
# Through Tor
|
|
s5p -C socks5://127.0.0.1:9050
|
|
|
|
# Through Tor + another proxy
|
|
s5p -C socks5://127.0.0.1:9050,socks5://proxy:1080
|
|
|
|
# Custom listen address
|
|
s5p -l 0.0.0.0:9999 -C socks5://127.0.0.1:9050
|
|
|
|
# From config file
|
|
s5p -c config/s5p.yaml
|
|
|
|
# With proxy source API (rotate exit proxy per-connection)
|
|
s5p -C socks5://127.0.0.1:9050 -S http://10.200.1.250:8081/proxies
|
|
|
|
# Debug mode
|
|
s5p -v -C socks5://127.0.0.1:9050
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Copy the tracked example to create your live config:
|
|
|
|
```bash
|
|
cp config/example.yaml config/s5p.yaml
|
|
```
|
|
|
|
| File | Tracked | Purpose |
|
|
|------|---------|---------|
|
|
| `config/example.yaml` | yes | Template with placeholder addresses |
|
|
| `config/s5p.yaml` | no (gitignored) | Live config with real proxy addresses |
|
|
|
|
```yaml
|
|
listen: 127.0.0.1:1080
|
|
timeout: 10
|
|
retries: 3
|
|
log_level: info
|
|
|
|
chain:
|
|
- socks5://127.0.0.1:9050
|
|
|
|
proxy_pool:
|
|
sources:
|
|
- url: http://10.200.1.250:8081/proxies
|
|
proto: socks5
|
|
limit: 1000
|
|
- file: /etc/s5p/proxies.txt
|
|
refresh: 300
|
|
test_interval: 120
|
|
test_url: http://httpbin.org/ip
|
|
test_timeout: 15
|
|
test_concurrency: 5
|
|
max_fails: 3
|
|
state_file: "" # empty = ~/.cache/s5p/pool.json
|
|
```
|
|
|
|
## Proxy URL Format
|
|
|
|
```
|
|
protocol://[username:password@]host[:port]
|
|
```
|
|
|
|
| Protocol | Default Port | Auth Support |
|
|
|----------|-------------|-------------|
|
|
| socks5 | 1080 | username/password |
|
|
| socks4 | 1080 | none |
|
|
| http | 8080 | Basic |
|
|
|
|
## Container
|
|
|
|
```bash
|
|
make build # build image
|
|
make up # start container (detached)
|
|
make logs # follow logs
|
|
make down # stop and remove container
|
|
```
|
|
|
|
Source (`./src`) and config (`./config/s5p.yaml`) are mounted read-only
|
|
into the container. Edit locally, restart to pick up changes.
|
|
|
|
## Proxy Pool
|
|
|
|
Managed proxy pool with multiple sources, health testing, and persistence.
|
|
Appends a random alive proxy after the static chain on each connection.
|
|
|
|
```yaml
|
|
proxy_pool:
|
|
sources:
|
|
- url: http://10.200.1.250:8081/proxies
|
|
proto: socks5 # optional: filter by protocol
|
|
country: US # optional: filter by country
|
|
limit: 1000 # max proxies to fetch from API
|
|
- file: /etc/s5p/proxies.txt # text file, one proxy URL per line
|
|
refresh: 300 # re-fetch sources every 300 seconds
|
|
test_interval: 120 # health test cycle every 120 seconds
|
|
test_url: http://httpbin.org/ip # URL for health checks
|
|
test_timeout: 15 # per-test timeout (seconds)
|
|
test_concurrency: 5 # parallel health tests
|
|
max_fails: 3 # evict after N consecutive failures
|
|
state_file: "" # empty = ~/.cache/s5p/pool.json
|
|
```
|
|
|
|
### Sources
|
|
|
|
| Type | Config key | Description |
|
|
|------|-----------|-------------|
|
|
| HTTP API | `url` | JSON: `{"proxies": [{"proto": "socks5", "proxy": "host:port"}, ...]}` |
|
|
| Text file | `file` | One proxy URL per line, `#` comments, blank lines ignored |
|
|
|
|
### Proxy file format
|
|
|
|
```
|
|
# Exit proxies
|
|
socks5://1.2.3.4:1080
|
|
socks5://user:pass@5.6.7.8:1080
|
|
http://proxy.example.com:8080
|
|
```
|
|
|
|
### Health testing
|
|
|
|
Each cycle tests all proxies through the full chain (static chain + proxy)
|
|
by sending an HTTP GET to `test_url`. Proxies are marked alive on `200` response.
|
|
After `max_fails` consecutive failures, a proxy is evicted.
|
|
|
|
Mass-failure guard: if >90% of tests fail in one cycle, eviction is skipped
|
|
(likely the static chain is broken, not the proxies).
|
|
|
|
### Persistence
|
|
|
|
Pool state is saved to `state_file` (default: `~/.cache/s5p/pool.json`) after
|
|
each refresh/health cycle and on shutdown. On startup, previously-alive proxies
|
|
are loaded for fast warm starts.
|
|
|
|
### CLI shorthand
|
|
|
|
```bash
|
|
s5p -C socks5://127.0.0.1:9050 -S http://10.200.1.250:8081/proxies
|
|
```
|
|
|
|
The `-S` flag creates a pool with a single API source (uses defaults for all
|
|
other pool settings).
|
|
|
|
### Legacy config
|
|
|
|
The old `proxy_source` key is still supported and auto-converts to `proxy_pool`
|
|
with a single API source. `proxy_pool` takes precedence if both are present.
|
|
|
|
## Connection Retry
|
|
|
|
When a proxy pool is active, s5p retries failed connections with a different
|
|
random proxy. Controlled by the `retries` setting (default: 3). Static-only
|
|
chains do not retry (retrying the same chain is pointless).
|
|
|
|
```yaml
|
|
retries: 5 # try up to 5 different proxies per connection
|
|
```
|
|
|
|
```bash
|
|
s5p -r 5 -C socks5://127.0.0.1:9050 -S http://api:8081/proxies
|
|
```
|
|
|
|
## Metrics
|
|
|
|
s5p tracks connection metrics and logs a summary every 60 seconds and on
|
|
shutdown:
|
|
|
|
```
|
|
metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s
|
|
```
|
|
|
|
| Counter | Meaning |
|
|
|---------|---------|
|
|
| `conn` | Total incoming connections |
|
|
| `ok` | Successfully connected + relayed |
|
|
| `fail` | All retries exhausted |
|
|
| `retries` | Total retry attempts |
|
|
| `active` | Currently relaying |
|
|
| `in` | Bytes client -> remote |
|
|
| `out` | Bytes remote -> client |
|
|
| `up` | Server uptime |
|
|
|
|
## Profiling
|
|
|
|
```bash
|
|
# Run with cProfile enabled
|
|
s5p --cprofile -c config/s5p.yaml
|
|
|
|
# Custom output file
|
|
s5p --cprofile output.prof -c config/s5p.yaml
|
|
|
|
# Analyze after stopping
|
|
python -m pstats s5p.prof
|
|
```
|
|
|
|
## Testing the Proxy
|
|
|
|
```bash
|
|
# Check exit IP via Tor
|
|
curl --proxy socks5h://127.0.0.1:1080 https://check.torproject.org/api/ip
|
|
|
|
# Fetch a page
|
|
curl --proxy socks5h://127.0.0.1:1080 https://example.com
|
|
|
|
# Use with Firefox: set SOCKS5 proxy to 127.0.0.1:1080, enable remote DNS
|
|
```
|
|
|
|
Note: use `socks5h://` (not `socks5://`) with curl to send DNS through the proxy.
|
|
|
|
## Chain Order
|
|
|
|
Hops are traversed left-to-right:
|
|
|
|
```
|
|
-C hop1,hop2,hop3
|
|
|
|
Client -> s5p -> hop1 -> hop2 -> hop3 -> Destination
|
|
```
|
|
|
|
Each hop only sees its immediate neighbors.
|