username/s5p
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: control API and Tor integration #1

Merged
username merged 9 commits from feat/control-api into main 2026-02-17 09:56:06 +00:00
Conversation 0 Commits 9 Files Changed 16 +89 -2
6 changed files with 89 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit c939101a73 - Show all commits

2
PROJECT.md
View File

@@ -25,6 +25,7 @@ Client -------> s5p -------> Hop 1 -------> Hop 2 -------> Target
- **pool.py** -- managed proxy pool (multi-source, health-tested, persistent)
- **http.py** -- minimal async HTTP/1.1 client (GET/POST JSON, no external deps)
- **connpool.py** -- pre-warmed TCP connection pool to first chain hop
- **api.py** -- built-in HTTP control API (runtime metrics, pool state, config reload)
- **cli.py** -- argparse CLI, logging setup, cProfile support
- **metrics.py** -- connection counters and human-readable summary (lock-free, asyncio-only)
@@ -66,3 +67,4 @@ All other functionality uses Python stdlib (`asyncio`, `socket`, `struct`).
- **Connection semaphore** -- cap concurrent connections to prevent fd exhaustion
- **Async HTTP** -- native asyncio HTTP client replaces blocking urllib, parallel fetches
- **First-hop pool** -- pre-warmed TCP connections to chain[0], stale-evicted, auto-refilled
- **Control API** -- built-in asyncio HTTP server, no Flask/external deps, disabled by default

5
README.md
View File

@@ -21,6 +21,7 @@ through configurable chains of SOCKS4, SOCKS5, and HTTP CONNECT proxies.
- Concurrent connection limit with backpressure (`max_connections`)
- Async HTTP client for proxy source fetching (parallel, no threads)
- First-hop TCP connection pool (pre-warmed, stale-evicted)
- Built-in control API (runtime metrics, pool state, config reload via HTTP)
- Container-ready (Alpine-based, podman/docker)
- Graceful shutdown (SIGTERM/SIGINT)
- Pure Python, asyncio-based, minimal dependencies
@@ -70,6 +71,7 @@ timeout: 10
retries: 3
max_connections: 256 # concurrent connection limit
pool_size: 8 # pre-warmed connections to first hop
api_listen: 127.0.0.1:1081 # control API (disabled by default)
chain:
- socks5://127.0.0.1:9050 # Tor
@@ -89,7 +91,7 @@ proxy_pool:
## CLI Reference
```
s5p [-c FILE] [-l [HOST:]PORT] [-C URL[,URL,...]] [-S URL] [-t SEC] [-r N] [-m N] [-v|-q]
s5p [-c FILE] [-l [HOST:]PORT] [-C URL[,URL,...]] [-S URL] [-t SEC] [-r N] [-m N] [--api [HOST:]PORT] [-v|-q]
Options:
-c, --config FILE YAML config file
@@ -99,6 +101,7 @@ Options:
-t, --timeout SEC Per-hop timeout (default: 10)
-r, --retries N Max attempts per connection (default: 3, proxy_source only)
-m, --max-connections N Max concurrent connections (default: 256)
--api [HOST:]PORT Enable control API (e.g. 127.0.0.1:1081)
-v, --verbose Debug logging
-q, --quiet Errors only
--cprofile [FILE] Enable cProfile, dump to FILE (default: s5p.prof)

1
ROADMAP.md
View File

@@ -22,6 +22,7 @@
## v0.2.0
- [x] Built-in control API (runtime metrics, pool state, config reload)
- [ ] SOCKS5 server authentication (username/password)
- [ ] Tor control port integration (circuit renewal via NEWNYM)
- [ ] Metrics (connections/sec, bytes relayed, hop latency)

1
TASKS.md
View File

@@ -42,6 +42,7 @@
- [x] Instant warm start (trust cached state, defer all health tests)
- [x] Register signal handlers before startup (fix SIGKILL on stop)
- [x] Use k8s-file logging driver with rotation
- [x] Built-in control API (`api.py`, `--api`, `api_listen`)
## Next
- [ ] Integration tests with mock proxy server

15
docs/CHEATSHEET.md
View File

@@ -14,6 +14,7 @@ s5p -q # errors only
s5p -S http://api:8081/proxies # proxy source API
s5p -r 5 # retry up to 5 proxies
s5p -m 512 # max concurrent connections
s5p --api 127.0.0.1:1081 # enable control API
s5p --cprofile # profile to s5p.prof
s5p --cprofile out.prof # profile to custom file
```
@@ -84,6 +85,20 @@ http://host:port
http://user:pass@host:port
```
## Control API
```bash
s5p --api 127.0.0.1:1081 -c config/s5p.yaml # enable API
curl -s http://127.0.0.1:1081/status | jq . # runtime status
curl -s http://127.0.0.1:1081/metrics | jq . # full metrics
curl -s http://127.0.0.1:1081/pool | jq . # all proxies
curl -s http://127.0.0.1:1081/pool/alive | jq . # alive only
curl -s http://127.0.0.1:1081/config | jq . # current config
curl -s -X POST http://127.0.0.1:1081/reload # reload config
curl -s -X POST http://127.0.0.1:1081/pool/test # health test now
```
## Testing
```bash

67
docs/USAGE.md
View File

@@ -46,6 +46,7 @@ log_level: info
max_connections: 256 # concurrent connection limit (backpressure)
pool_size: 0 # pre-warmed TCP connections to first hop (0 = disabled)
pool_max_idle: 30 # max idle time for pooled connections (seconds)
api_listen: "" # control API bind address (empty = disabled)
chain:
- socks5://127.0.0.1:9050
@@ -216,6 +217,70 @@ other pool settings).
The old `proxy_source` key is still supported and auto-converts to `proxy_pool`
with a single API source. `proxy_pool` takes precedence if both are present.
## Control API
Built-in HTTP API for runtime inspection and management. Disabled by default;
enable with `api_listen` in config or `--api` on the command line.
```yaml
api_listen: 127.0.0.1:1081
```
```bash
s5p --api 127.0.0.1:1081 -c config/s5p.yaml
```
### Read endpoints
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/status` | Combined summary: uptime, metrics, pool stats, chain |
| `GET` | `/metrics` | Full metrics counters (connections, bytes, uptime) |
| `GET` | `/pool` | All proxies with per-entry state |
| `GET` | `/pool/alive` | Alive proxies only |
| `GET` | `/config` | Current runtime config (sanitized) |
### Write endpoints
| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/reload` | Re-read config file (replaces SIGHUP) |
| `POST` | `/pool/test` | Trigger immediate health test cycle |
| `POST` | `/pool/refresh` | Trigger immediate source re-fetch |
All responses are `application/json`. Errors return `{"error": "message"}` with
appropriate status code (400, 404, 405, 500).
### Examples
```bash
# Runtime status
curl -s http://127.0.0.1:1081/status | jq .
# Full metrics
curl -s http://127.0.0.1:1081/metrics | jq .
# Pool state (all proxies)
curl -s http://127.0.0.1:1081/pool | jq .
# Alive proxies only
curl -s http://127.0.0.1:1081/pool/alive | jq '.proxies | length'
# Current config
curl -s http://127.0.0.1:1081/config | jq .
# Reload config (like SIGHUP)
curl -s -X POST http://127.0.0.1:1081/reload | jq .
# Trigger health tests now
curl -s -X POST http://127.0.0.1:1081/pool/test | jq .
# Re-fetch proxy sources now
curl -s -X POST http://127.0.0.1:1081/pool/refresh | jq .
```
Settings that require a restart: `listen`, `chain`, `pool_size`, `pool_max_idle`, `api_listen`.
## Connection Retry
When a proxy pool is active, s5p retries failed connections with a different
@@ -250,7 +315,7 @@ Settings reloaded on SIGHUP:
| `max_connections` | Concurrent connection limit |
| `proxy_pool.*` | Sources, intervals, thresholds |
Settings that require a restart: `listen`, `chain`, `pool_size`, `pool_max_idle`.
Settings that require a restart: `listen`, `chain`, `pool_size`, `pool_max_idle`, `api_listen`.
Requires `-c` / `--config` to know which file to re-read. Without a
config file, SIGHUP is ignored with a warning.