diff --git a/PROJECT.md b/PROJECT.md index 0ef26f8..a94434e 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -25,6 +25,7 @@ Client -------> s5p -------> Hop 1 -------> Hop 2 -------> Target - **pool.py** -- managed proxy pool (multi-source, health-tested, persistent) - **http.py** -- minimal async HTTP/1.1 client (GET/POST JSON, no external deps) - **connpool.py** -- pre-warmed TCP connection pool to first chain hop +- **api.py** -- built-in HTTP control API (runtime metrics, pool state, config reload) - **cli.py** -- argparse CLI, logging setup, cProfile support - **metrics.py** -- connection counters and human-readable summary (lock-free, asyncio-only) @@ -66,3 +67,4 @@ All other functionality uses Python stdlib (`asyncio`, `socket`, `struct`). - **Connection semaphore** -- cap concurrent connections to prevent fd exhaustion - **Async HTTP** -- native asyncio HTTP client replaces blocking urllib, parallel fetches - **First-hop pool** -- pre-warmed TCP connections to chain[0], stale-evicted, auto-refilled +- **Control API** -- built-in asyncio HTTP server, no Flask/external deps, disabled by default diff --git a/README.md b/README.md index fe6337a..d5ecbe0 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,7 @@ through configurable chains of SOCKS4, SOCKS5, and HTTP CONNECT proxies. - Concurrent connection limit with backpressure (`max_connections`) - Async HTTP client for proxy source fetching (parallel, no threads) - First-hop TCP connection pool (pre-warmed, stale-evicted) +- Built-in control API (runtime metrics, pool state, config reload via HTTP) - Container-ready (Alpine-based, podman/docker) - Graceful shutdown (SIGTERM/SIGINT) - Pure Python, asyncio-based, minimal dependencies @@ -70,6 +71,7 @@ timeout: 10 retries: 3 max_connections: 256 # concurrent connection limit pool_size: 8 # pre-warmed connections to first hop +api_listen: 127.0.0.1:1081 # control API (disabled by default) chain: - socks5://127.0.0.1:9050 # Tor @@ -89,7 +91,7 @@ proxy_pool: ## CLI Reference ``` -s5p [-c FILE] [-l [HOST:]PORT] [-C URL[,URL,...]] [-S URL] [-t SEC] [-r N] [-m N] [-v|-q] +s5p [-c FILE] [-l [HOST:]PORT] [-C URL[,URL,...]] [-S URL] [-t SEC] [-r N] [-m N] [--api [HOST:]PORT] [-v|-q] Options: -c, --config FILE YAML config file @@ -99,6 +101,7 @@ Options: -t, --timeout SEC Per-hop timeout (default: 10) -r, --retries N Max attempts per connection (default: 3, proxy_source only) -m, --max-connections N Max concurrent connections (default: 256) + --api [HOST:]PORT Enable control API (e.g. 127.0.0.1:1081) -v, --verbose Debug logging -q, --quiet Errors only --cprofile [FILE] Enable cProfile, dump to FILE (default: s5p.prof) diff --git a/ROADMAP.md b/ROADMAP.md index a66082e..ac5f329 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -22,6 +22,7 @@ ## v0.2.0 +- [x] Built-in control API (runtime metrics, pool state, config reload) - [ ] SOCKS5 server authentication (username/password) - [ ] Tor control port integration (circuit renewal via NEWNYM) - [ ] Metrics (connections/sec, bytes relayed, hop latency) diff --git a/TASKS.md b/TASKS.md index 1154a94..3bb79c2 100644 --- a/TASKS.md +++ b/TASKS.md @@ -42,6 +42,7 @@ - [x] Instant warm start (trust cached state, defer all health tests) - [x] Register signal handlers before startup (fix SIGKILL on stop) - [x] Use k8s-file logging driver with rotation +- [x] Built-in control API (`api.py`, `--api`, `api_listen`) ## Next - [ ] Integration tests with mock proxy server diff --git a/docs/CHEATSHEET.md b/docs/CHEATSHEET.md index a945474..bff5a6d 100644 --- a/docs/CHEATSHEET.md +++ b/docs/CHEATSHEET.md @@ -14,6 +14,7 @@ s5p -q # errors only s5p -S http://api:8081/proxies # proxy source API s5p -r 5 # retry up to 5 proxies s5p -m 512 # max concurrent connections +s5p --api 127.0.0.1:1081 # enable control API s5p --cprofile # profile to s5p.prof s5p --cprofile out.prof # profile to custom file ``` @@ -84,6 +85,20 @@ http://host:port http://user:pass@host:port ``` +## Control API + +```bash +s5p --api 127.0.0.1:1081 -c config/s5p.yaml # enable API + +curl -s http://127.0.0.1:1081/status | jq . # runtime status +curl -s http://127.0.0.1:1081/metrics | jq . # full metrics +curl -s http://127.0.0.1:1081/pool | jq . # all proxies +curl -s http://127.0.0.1:1081/pool/alive | jq . # alive only +curl -s http://127.0.0.1:1081/config | jq . # current config +curl -s -X POST http://127.0.0.1:1081/reload # reload config +curl -s -X POST http://127.0.0.1:1081/pool/test # health test now +``` + ## Testing ```bash diff --git a/docs/USAGE.md b/docs/USAGE.md index f5f548e..3a05155 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -46,6 +46,7 @@ log_level: info max_connections: 256 # concurrent connection limit (backpressure) pool_size: 0 # pre-warmed TCP connections to first hop (0 = disabled) pool_max_idle: 30 # max idle time for pooled connections (seconds) +api_listen: "" # control API bind address (empty = disabled) chain: - socks5://127.0.0.1:9050 @@ -216,6 +217,70 @@ other pool settings). The old `proxy_source` key is still supported and auto-converts to `proxy_pool` with a single API source. `proxy_pool` takes precedence if both are present. +## Control API + +Built-in HTTP API for runtime inspection and management. Disabled by default; +enable with `api_listen` in config or `--api` on the command line. + +```yaml +api_listen: 127.0.0.1:1081 +``` + +```bash +s5p --api 127.0.0.1:1081 -c config/s5p.yaml +``` + +### Read endpoints + +| Method | Path | Description | +|--------|------|-------------| +| `GET` | `/status` | Combined summary: uptime, metrics, pool stats, chain | +| `GET` | `/metrics` | Full metrics counters (connections, bytes, uptime) | +| `GET` | `/pool` | All proxies with per-entry state | +| `GET` | `/pool/alive` | Alive proxies only | +| `GET` | `/config` | Current runtime config (sanitized) | + +### Write endpoints + +| Method | Path | Description | +|--------|------|-------------| +| `POST` | `/reload` | Re-read config file (replaces SIGHUP) | +| `POST` | `/pool/test` | Trigger immediate health test cycle | +| `POST` | `/pool/refresh` | Trigger immediate source re-fetch | + +All responses are `application/json`. Errors return `{"error": "message"}` with +appropriate status code (400, 404, 405, 500). + +### Examples + +```bash +# Runtime status +curl -s http://127.0.0.1:1081/status | jq . + +# Full metrics +curl -s http://127.0.0.1:1081/metrics | jq . + +# Pool state (all proxies) +curl -s http://127.0.0.1:1081/pool | jq . + +# Alive proxies only +curl -s http://127.0.0.1:1081/pool/alive | jq '.proxies | length' + +# Current config +curl -s http://127.0.0.1:1081/config | jq . + +# Reload config (like SIGHUP) +curl -s -X POST http://127.0.0.1:1081/reload | jq . + +# Trigger health tests now +curl -s -X POST http://127.0.0.1:1081/pool/test | jq . + +# Re-fetch proxy sources now +curl -s -X POST http://127.0.0.1:1081/pool/refresh | jq . +``` + +Settings that require a restart: `listen`, `chain`, `pool_size`, `pool_max_idle`, `api_listen`. + ## Connection Retry When a proxy pool is active, s5p retries failed connections with a different @@ -250,7 +315,7 @@ Settings reloaded on SIGHUP: | `max_connections` | Concurrent connection limit | | `proxy_pool.*` | Sources, intervals, thresholds | -Settings that require a restart: `listen`, `chain`, `pool_size`, `pool_max_idle`. +Settings that require a restart: `listen`, `chain`, `pool_size`, `pool_max_idle`, `api_listen`. Requires `-c` / `--config` to know which file to re-read. Without a config file, SIGHUP is ignored with a warning.