s5p/docs/USAGE.md

# s5p -- Usage

## Basic Usage

```bash
# Direct proxy (no chain, just a SOCKS5 server)
s5p

# Through Tor
s5p -C socks5://127.0.0.1:9050

# Through Tor + another proxy
s5p -C socks5://127.0.0.1:9050,socks5://proxy:1080

# Custom listen address
s5p -l 0.0.0.0:9999 -C socks5://127.0.0.1:9050

# From config file
s5p -c config/s5p.yaml

# With proxy source API (rotate exit proxy per-connection)
s5p -C socks5://127.0.0.1:9050 -S http://10.200.1.250:8081/proxies

# Debug mode
s5p -v -C socks5://127.0.0.1:9050
```

## Configuration

Copy the tracked example to create your live config:

```bash
cp config/example.yaml config/s5p.yaml
```

| File | Tracked | Purpose |
|------|---------|---------|
| `config/example.yaml` | yes | Template with placeholder addresses |
| `config/s5p.yaml` | no (gitignored) | Live config with real proxy addresses |

```yaml
listen: 127.0.0.1:1080
timeout: 10
retries: 3
log_level: info

chain:
  - socks5://127.0.0.1:9050

proxy_pool:
  sources:
    - url: http://10.200.1.250:8081/proxies
      proto: socks5
      limit: 1000
    - file: /etc/s5p/proxies.txt
  refresh: 300
  test_interval: 120
  test_url: http://httpbin.org/ip
  test_timeout: 15
  test_concurrency: 5
  max_fails: 3
  state_file: ""      # empty = ~/.cache/s5p/pool.json
```

## Proxy URL Format

```
protocol://[username:password@]host[:port]
```

| Protocol | Default Port | Auth Support |
|----------|-------------|-------------|
| socks5   | 1080        | username/password |
| socks4   | 1080        | none |
| http     | 8080        | Basic |

## Container

```bash
make build   # build image
make up      # start container (detached)
make logs    # follow logs
make down    # stop and remove container
```

Source (`./src`) and config (`./config/s5p.yaml`) are mounted read-only
into the container. Edit locally, restart to pick up changes.

## Proxy Pool

Managed proxy pool with multiple sources, health testing, and persistence.
Appends an alive proxy after the static chain on each connection, weighted
by recency of last successful health test.

```yaml
proxy_pool:
  sources:
    - url: http://10.200.1.250:8081/proxies
      proto: socks5         # optional: filter by protocol
      country: US           # optional: filter by country
      limit: 1000           # max proxies to fetch from API
    - file: /etc/s5p/proxies.txt  # text file, one proxy URL per line
  refresh: 300              # re-fetch sources every 300 seconds
  test_interval: 120        # health test cycle every 120 seconds
  test_url: http://httpbin.org/ip  # URL for health checks
  test_timeout: 15          # per-test timeout (seconds)
  test_concurrency: 5       # parallel health tests
  max_fails: 3              # evict after N consecutive failures
  state_file: ""            # empty = ~/.cache/s5p/pool.json
```

### Sources

| Type | Config key | Description |
|------|-----------|-------------|
| HTTP API | `url` | JSON: `{"proxies": [{"proto": "socks5", "proxy": "host:port"}, ...]}` |
| Text file | `file` | One proxy URL per line, `#` comments, blank lines ignored |

### Proxy file format

```
# Exit proxies
socks5://1.2.3.4:1080
socks5://user:pass@5.6.7.8:1080
http://proxy.example.com:8080
```

### Health testing

Each cycle tests all proxies through the full chain (static chain + proxy)
by sending an HTTP GET to `test_url`. Proxies are marked alive on `200` response.
After `max_fails` consecutive failures, a proxy is evicted.

Before each health test cycle, the static chain is tested without any pool
proxy. If the chain itself is unreachable (e.g., Tor is down), proxy tests
are skipped entirely and a warning is logged. This prevents false mass-failure
and unnecessary evictions.

Mass-failure guard: if >90% of tests fail in one cycle, eviction is skipped
(likely the static chain is broken, not the proxies).

### Selection weight

Alive proxies are selected with probability proportional to their recency
weight: `1 / (1 + age / 300)`, where `age` is seconds since the last
successful health test. This favors freshly-verified proxies over stale ones:

| Age | Weight |
|-----|--------|
| 0 s (just tested) | ~1.0 |
| 5 min | ~0.5 |
| 10 min | ~0.33 |
| 30 min | ~0.1 |
| Never tested | 0.01 |

### Failure backoff

When a proxy fails during an actual connection attempt (not just a health
test), its weight is penalized for 60 seconds. The penalty ramps linearly
from floor (0.01) back to normal over that window. This prevents retries
from repeatedly selecting a proxy that just failed.

### Stale proxy expiry

Proxies not returned by any source for 3 consecutive refresh cycles and
not currently alive are automatically evicted. This cleans up proxies
removed upstream faster than waiting for `max_fails` health test failures.

### Persistence

Pool state is saved to `state_file` (default: `~/.cache/s5p/pool.json`) after
each refresh/health cycle and on shutdown. On startup, previously-alive proxies
are loaded for fast warm starts.

### Warm start

When restarting with an existing state file, only the previously-alive proxies
are tested before the server starts accepting connections. A full health test
of all proxies runs in the background. This reduces startup blocking from
minutes to seconds on warm restarts. Cold starts (no state file) test all
proxies before serving.

### CLI shorthand

```bash
s5p -C socks5://127.0.0.1:9050 -S http://10.200.1.250:8081/proxies
```

The `-S` flag creates a pool with a single API source (uses defaults for all
other pool settings).

### Legacy config

The old `proxy_source` key is still supported and auto-converts to `proxy_pool`
with a single API source. `proxy_pool` takes precedence if both are present.

## Connection Retry

When a proxy pool is active, s5p retries failed connections with a different
random proxy. Controlled by the `retries` setting (default: 3). Static-only
chains do not retry (retrying the same chain is pointless).

```yaml
retries: 5   # try up to 5 different proxies per connection
```

```bash
s5p -r 5 -C socks5://127.0.0.1:9050 -S http://api:8081/proxies
```

## Metrics

s5p tracks connection metrics and logs a summary every 60 seconds and on
shutdown:

```
metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s pool=42/65
```

| Counter | Meaning |
|---------|---------|
| `conn` | Total incoming connections |
| `ok` | Successfully connected + relayed |
| `fail` | All retries exhausted |
| `retries` | Total retry attempts |
| `active` | Currently relaying |
| `in` | Bytes client -> remote |
| `out` | Bytes remote -> client |
| `up` | Server uptime |
| `pool` | Alive/total proxies (only when pool is active) |

## Profiling

```bash
# Run with cProfile enabled
s5p --cprofile -c config/s5p.yaml

# Custom output file
s5p --cprofile output.prof -c config/s5p.yaml

# Analyze after stopping
python -m pstats s5p.prof
```

## Testing the Proxy

```bash
# Check exit IP via Tor
curl --proxy socks5h://127.0.0.1:1080 https://check.torproject.org/api/ip

# Fetch a page
curl --proxy socks5h://127.0.0.1:1080 https://example.com

# Use with Firefox: set SOCKS5 proxy to 127.0.0.1:1080, enable remote DNS
```

Note: use `socks5h://` (not `socks5://`) with curl to send DNS through the proxy.

## Chain Order

Hops are traversed left-to-right:

```
-C hop1,hop2,hop3

Client -> s5p -> hop1 -> hop2 -> hop3 -> Destination
```

Each hop only sees its immediate neighbors.