Compare commits
14 Commits
94e91d9e27
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0ae7b13407 | ||
|
|
a1996b1c9e | ||
|
|
051c0ac719 | ||
|
|
b4cf4fc8ae | ||
|
|
251d99795b | ||
|
|
cca76d4974 | ||
|
|
a64b09de8e | ||
|
|
de5f586bc7 | ||
|
|
3e2c431f49 | ||
|
|
9a56dc778e | ||
|
|
3593481b30 | ||
|
|
c1c92ddc39 | ||
|
|
a741c0a017 | ||
|
|
53fdc4527f |
@@ -6,38 +6,43 @@ on:
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: linux
|
||||
container: python:3.13-alpine
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.13'
|
||||
- run: pip install pyyaml ruff pytest
|
||||
- run: apk add --no-cache git
|
||||
- run: |
|
||||
git clone --depth 1 \
|
||||
-c "http.extraHeader=Authorization: token ${{ github.token }}" \
|
||||
"${{ github.server_url }}/${{ github.repository }}.git" .
|
||||
- run: pip install --no-cache-dir -r requirements.txt ruff pytest
|
||||
- run: ruff check src/ tests/
|
||||
- run: PYTHONPATH=src pytest tests/ -v
|
||||
|
||||
secrets:
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: linux
|
||||
container: ghcr.io/gitleaks/gitleaks:latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- run: |
|
||||
docker run --rm \
|
||||
-v "$PWD:/scan:ro" \
|
||||
ghcr.io/gitleaks/gitleaks:latest \
|
||||
detect --source /scan -v
|
||||
git clone \
|
||||
-c "http.extraHeader=Authorization: token ${{ github.token }}" \
|
||||
"${{ github.server_url }}/${{ github.repository }}.git" .
|
||||
- run: gitleaks detect --source . -v
|
||||
|
||||
build:
|
||||
needs: [test, secrets]
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: linux
|
||||
container: quay.io/podman/stable
|
||||
env:
|
||||
CONTAINER_HOST: unix:///var/run/docker.sock
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- run: dnf install -y git
|
||||
- run: |
|
||||
mkdir -p ~/.docker
|
||||
AUTH=$(printf '%s:%s' "$HARBOR_USER" "$HARBOR_PASS" | base64 -w0)
|
||||
printf '{"auths":{"harbor.mymx.me":{"auth":"%s"}}}\n' "$AUTH" > ~/.docker/config.json
|
||||
git clone --depth 1 \
|
||||
-c "http.extraHeader=Authorization: token ${{ github.token }}" \
|
||||
"${{ github.server_url }}/${{ github.repository }}.git" .
|
||||
- run: echo "$HARBOR_PASS" | podman --remote login -u "$HARBOR_USER" --password-stdin harbor.mymx.me
|
||||
env:
|
||||
HARBOR_USER: ${{ secrets.HARBOR_USER }}
|
||||
HARBOR_PASS: ${{ secrets.HARBOR_PASS }}
|
||||
- run: docker build --push -t harbor.mymx.me/s5p/s5p:latest -f Containerfile .
|
||||
- run: podman --remote build -t harbor.mymx.me/s5p/s5p:latest -f Containerfile .
|
||||
- run: podman --remote push harbor.mymx.me/s5p/s5p:latest
|
||||
|
||||
5
.gitleaks.toml
Normal file
5
.gitleaks.toml
Normal file
@@ -0,0 +1,5 @@
|
||||
[allowlist]
|
||||
paths = [
|
||||
'''tests/''',
|
||||
'''docs/''',
|
||||
]
|
||||
@@ -1,9 +1,11 @@
|
||||
FROM python:3.13-alpine
|
||||
|
||||
RUN pip install --no-cache-dir pyyaml>=6.0
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir --upgrade pip && \
|
||||
pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
ENV PYTHONUNBUFFERED=1 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONPATH=/app/src
|
||||
|
||||
9
Makefile
9
Makefile
@@ -1,10 +1,17 @@
|
||||
APP_NAME := s5p
|
||||
|
||||
.PHONY: install test lint clean build up down logs
|
||||
.PHONY: install install-service test lint clean build up down logs
|
||||
|
||||
install:
|
||||
pip install -e .
|
||||
|
||||
install-service:
|
||||
sudo mkdir -p /etc/s5p
|
||||
sudo cp config/s5p.service /etc/systemd/system/s5p.service
|
||||
sudo systemctl daemon-reload
|
||||
@echo "Unit installed. Configure /etc/s5p/s5p.yaml, then:"
|
||||
@echo " sudo systemctl enable --now s5p"
|
||||
|
||||
test:
|
||||
pytest tests/ -v
|
||||
|
||||
|
||||
16
ROADMAP.md
16
ROADMAP.md
@@ -1,6 +1,6 @@
|
||||
# s5p -- Roadmap
|
||||
|
||||
## v0.1.0 (current)
|
||||
## v0.1.0
|
||||
|
||||
- [x] SOCKS5 server (CONNECT command)
|
||||
- [x] Proxy chaining (SOCKS5, SOCKS4/4a, HTTP CONNECT)
|
||||
@@ -32,9 +32,17 @@
|
||||
- [x] Named proxy pools with per-listener assignment (`proxy_pools:`)
|
||||
- [x] MITM source filter (`mitm: true/false` on pool sources)
|
||||
|
||||
## v0.3.0
|
||||
## v0.3.0 (current)
|
||||
|
||||
- [ ] SOCKS5 server authentication (username/password)
|
||||
- [x] SOCKS5 server authentication (username/password)
|
||||
- [x] Systemd service unit
|
||||
- [x] CLI test coverage
|
||||
- [x] Protocol test coverage (SOCKS5/4/HTTP handshakes)
|
||||
- [x] API documentation (full response schemas)
|
||||
- [x] Prometheus metrics endpoint (`/metrics` OpenMetrics format)
|
||||
- [x] Listener-level retry override
|
||||
- [x] Pool-level proxy protocol filter (`allowed_protos`)
|
||||
- [x] Connection pooling documentation
|
||||
- [ ] UDP ASSOCIATE support (SOCKS5 UDP relay)
|
||||
- [ ] BIND support
|
||||
- [ ] Chain randomization (random order, random subset)
|
||||
@@ -42,6 +50,4 @@
|
||||
## v1.0.0
|
||||
|
||||
- [ ] Stable API and config format
|
||||
- [ ] Comprehensive test suite with mock proxies (integration tests done)
|
||||
- [ ] Systemd service unit
|
||||
- [ ] Performance benchmarks
|
||||
|
||||
18
TASKS.md
18
TASKS.md
@@ -67,6 +67,20 @@
|
||||
|
||||
- [x] Gitea CI workflow (lint + test + Harbor image push)
|
||||
|
||||
## v0.3.0 Stabilization
|
||||
- [x] Version bump to 0.3.0
|
||||
- [x] Systemd service unit (`config/s5p.service`, `make install-service`)
|
||||
- [x] CLI argument parsing tests (`tests/test_cli.py`)
|
||||
- [x] Protocol handshake tests (`tests/test_proto.py` -- SOCKS5/4/HTTP)
|
||||
- [x] API reference documentation (`docs/USAGE.md`)
|
||||
- [x] Prometheus `/metrics` endpoint (OpenMetrics format)
|
||||
|
||||
## Quick Wins
|
||||
- [x] Listener-level retry override (`retries` per listener)
|
||||
- [x] Pool-level proxy protocol filter (`allowed_protos` on proxy pool)
|
||||
- [x] Document connection pooling (`pool_size`/`pool_max_idle` in CHEATSHEET.md)
|
||||
|
||||
## Next
|
||||
- [x] Integration tests with mock proxy server
|
||||
- [ ] SOCKS5 server-side authentication
|
||||
- [ ] UDP ASSOCIATE support
|
||||
- [ ] BIND support
|
||||
- [ ] Chain randomization
|
||||
|
||||
10
TODO.md
10
TODO.md
@@ -6,24 +6,24 @@
|
||||
- Chain randomization modes (round-robin, sticky-per-destination)
|
||||
- Systemd socket activation
|
||||
- Per-pool health test chain override (different base chain per pool)
|
||||
- Pool-level proxy protocol filter (only socks5 from pool X, only http from pool Y)
|
||||
- Listener-level retry override (different retry count per listener)
|
||||
- ~~Pool-level proxy protocol filter (only socks5 from pool X, only http from pool Y)~~ (done)
|
||||
- ~~Listener-level retry override (different retry count per listener)~~ (done)
|
||||
|
||||
## Performance
|
||||
|
||||
- Benchmark relay throughput vs direct connection
|
||||
- Tune buffer sizes for different workloads
|
||||
- Connection pooling for frequently-used chains
|
||||
- ~~Connection pooling for frequently-used chains~~ (done: `pool_size`/`pool_max_idle`)
|
||||
|
||||
## Security
|
||||
|
||||
- Optional SOCKS5 server authentication
|
||||
- ~~Optional SOCKS5 server authentication~~ (done: fa36218)
|
||||
- Rate limiting per source IP
|
||||
- Access control lists
|
||||
|
||||
## Observability
|
||||
|
||||
- Prometheus metrics endpoint (`/metrics` in OpenMetrics format)
|
||||
- ~~Prometheus metrics endpoint (`/metrics` in OpenMetrics format)~~ (done)
|
||||
- Per-pool health test success rate tracking
|
||||
- Per-pool latency breakdown in `/status`
|
||||
|
||||
|
||||
@@ -28,11 +28,12 @@ chain:
|
||||
# sources:
|
||||
# - url: http://10.200.1.250:8081/proxies/all
|
||||
# mitm: false # filter: mitm=0 query param
|
||||
# allowed_protos: [socks5] # only accept socks5 from sources
|
||||
# state_file: /data/pool-clean.json
|
||||
# refresh: 300
|
||||
# test_interval: 120
|
||||
# test_timeout: 8
|
||||
# max_fails: 3
|
||||
# test_timeout: 12
|
||||
# max_fails: 5
|
||||
# mitm: # MITM-capable proxies
|
||||
# sources:
|
||||
# - url: http://10.200.1.250:8081/proxies/all
|
||||
@@ -40,8 +41,8 @@ chain:
|
||||
# state_file: /data/pool-mitm.json
|
||||
# refresh: 300
|
||||
# test_interval: 120
|
||||
# test_timeout: 8
|
||||
# max_fails: 3
|
||||
# test_timeout: 12
|
||||
# max_fails: 5
|
||||
|
||||
# Single proxy pool (legacy, still supported -- becomes pool "default"):
|
||||
# proxy_pool:
|
||||
@@ -74,8 +75,11 @@ chain:
|
||||
# newnym_interval: 0 # periodic NEWNYM (seconds, 0 = manual only)
|
||||
|
||||
# Multi-Tor round-robin -- distribute traffic across multiple Tor nodes.
|
||||
# When present, the first hop in each listener's chain is replaced at
|
||||
# connection time by round-robin selection from this list.
|
||||
# When present, the first hop in each listener's chain is REPLACED at
|
||||
# connection time by round-robin selection from this list. The first hop
|
||||
# specified in each listener's chain acts as a fallback only; tor_nodes
|
||||
# takes precedence for both client traffic and pool health tests.
|
||||
# Connection pools are pre-warmed for every node listed here.
|
||||
# tor_nodes:
|
||||
# - socks5://10.200.1.1:9050
|
||||
# - socks5://10.200.1.254:9050
|
||||
@@ -109,12 +113,13 @@ chain:
|
||||
# - localhost # exact hostname
|
||||
# - .local # domain suffix
|
||||
# chain:
|
||||
# - socks5://127.0.0.1:9050
|
||||
# - socks5://127.0.0.1:9050 # first hop (overridden by tor_nodes)
|
||||
# - [pool:clean, pool:mitm] # random choice per connection
|
||||
# - [pool:clean, pool:mitm] # independent random choice
|
||||
#
|
||||
# - listen: 0.0.0.0:1081
|
||||
# pool: clean
|
||||
# retries: 5 # override global retries for this listener
|
||||
# chain:
|
||||
# - socks5://127.0.0.1:9050
|
||||
# - pool # bare: uses default "clean"
|
||||
|
||||
15
config/s5p.service
Normal file
15
config/s5p.service
Normal file
@@ -0,0 +1,15 @@
|
||||
[Unit]
|
||||
Description=s5p SOCKS5 proxy
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/usr/local/bin/s5p -c /etc/s5p/s5p.yaml
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
KillSignal=SIGTERM
|
||||
TimeoutStopSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -21,6 +21,18 @@ s5p --tracemalloc # memory profile (top 10)
|
||||
s5p --tracemalloc 20 # memory profile (top 20)
|
||||
```
|
||||
|
||||
## Systemd
|
||||
|
||||
```
|
||||
make install-service # install unit + reload
|
||||
sudo systemctl enable --now s5p # enable + start
|
||||
sudo systemctl status s5p # check status
|
||||
sudo systemctl restart s5p # restart
|
||||
sudo systemctl stop s5p # stop
|
||||
journalctl -u s5p -f # follow logs
|
||||
journalctl -u s5p --since "5 min ago" # recent logs
|
||||
```
|
||||
|
||||
## Container
|
||||
|
||||
```
|
||||
@@ -120,6 +132,37 @@ curl -x socks5h://alice:s3cret@127.0.0.1:1080 https://example.com
|
||||
|
||||
No `auth:` key = no authentication required (default).
|
||||
|
||||
## Listener Retry Override (config)
|
||||
|
||||
```yaml
|
||||
listeners:
|
||||
- listen: 0.0.0.0:1080
|
||||
retries: 5 # override global retries
|
||||
chain:
|
||||
- socks5://127.0.0.1:9050
|
||||
- pool
|
||||
- listen: 0.0.0.0:1082
|
||||
chain:
|
||||
- socks5://127.0.0.1:9050 # 0 = use global default
|
||||
```
|
||||
|
||||
Per-listener `retries` overrides the global `retries` setting. Set to 0 (or
|
||||
omit) to inherit the global value.
|
||||
|
||||
## Pool Protocol Filter (config)
|
||||
|
||||
```yaml
|
||||
proxy_pools:
|
||||
socks_only:
|
||||
allowed_protos: [socks5] # reject http proxies
|
||||
sources:
|
||||
- url: http://api:8081/proxies/all
|
||||
```
|
||||
|
||||
When set, proxies not matching `allowed_protos` are silently dropped during
|
||||
merge. Useful when a source returns mixed protocols but the pool should
|
||||
only serve a specific type.
|
||||
|
||||
## Multi-Tor Round-Robin (config)
|
||||
|
||||
```yaml
|
||||
@@ -138,6 +181,23 @@ pool_size: 8 # pre-warmed TCP conns to first hop (0 = off)
|
||||
pool_max_idle: 30 # evict idle pooled conns (seconds)
|
||||
```
|
||||
|
||||
## Connection Pool (config)
|
||||
|
||||
```yaml
|
||||
pool_size: 8 # pre-warmed TCP connections per first hop (0 = off)
|
||||
pool_max_idle: 30 # evict idle connections after N seconds
|
||||
```
|
||||
|
||||
Pre-warms TCP connections to the first hop in the chain. Only the raw TCP
|
||||
connection is pooled -- SOCKS/HTTP negotiation consumes it. One pool is
|
||||
created per unique first hop (shared across listeners). Requires at least
|
||||
one hop in `chain`.
|
||||
|
||||
| Setting | Default | Notes |
|
||||
|---------|---------|-------|
|
||||
| `pool_size` | 0 (off) | Connections per first hop |
|
||||
| `pool_max_idle` | 30 | Idle eviction in seconds |
|
||||
|
||||
## Named Proxy Pools (config)
|
||||
|
||||
```yaml
|
||||
@@ -217,7 +277,7 @@ http://user:pass@host:port
|
||||
s5p --api 127.0.0.1:1081 -c config/s5p.yaml # enable API
|
||||
|
||||
curl -s http://127.0.0.1:1081/status | jq . # runtime status
|
||||
curl -s http://127.0.0.1:1081/metrics | jq . # full metrics
|
||||
curl -s http://127.0.0.1:1081/metrics # prometheus metrics
|
||||
curl -s http://127.0.0.1:1081/pool | jq . # all proxies
|
||||
curl -s http://127.0.0.1:1081/pool/alive | jq . # alive only
|
||||
curl -s http://127.0.0.1:1081/config | jq . # current config
|
||||
@@ -255,27 +315,26 @@ python -m pstats ~/.cache/s5p/s5p.prof # container profile output
|
||||
metrics: conn=1842 ok=1790 fail=52 retries=67 active=3 in=50.0M out=1.0G rate=4.72/s p50=198.3ms p95=890.1ms up=1h01m01s pool=42/65
|
||||
```
|
||||
|
||||
## Metrics JSON (`/metrics`)
|
||||
## Prometheus Metrics (`/metrics`)
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:1081/metrics | jq .
|
||||
curl -s http://127.0.0.1:1081/metrics
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"connections": 1842,
|
||||
"success": 1790,
|
||||
"rate": 4.72,
|
||||
"latency": {"count": 1000, "min": 45.2, "max": 2841.7, "avg": 312.4, "p50": 198.3, "p95": 890.1, "p99": 1523.6},
|
||||
"listener_latency": {
|
||||
"0.0.0.0:1080": {"count": 500, "p50": 1800.2, "p95": 8200.1, "...": "..."},
|
||||
"0.0.0.0:1081": {"count": 300, "p50": 1000.1, "p95": 3500.2, "...": "..."},
|
||||
"0.0.0.0:1082": {"count": 200, "p50": 400.1, "p95": 1200.5, "...": "..."}
|
||||
}
|
||||
}
|
||||
```
|
||||
# TYPE s5p_connections counter
|
||||
s5p_connections_total 1842
|
||||
# TYPE s5p_active_connections gauge
|
||||
s5p_active_connections 3
|
||||
# TYPE s5p_pool_proxies_alive gauge
|
||||
s5p_pool_proxies_alive{pool="clean"} 30
|
||||
# TYPE s5p_chain_latency_seconds summary
|
||||
s5p_chain_latency_seconds{quantile="0.5"} 0.198300
|
||||
s5p_chain_latency_seconds{quantile="0.95"} 0.890100
|
||||
# EOF
|
||||
```
|
||||
|
||||
Per-listener latency also appears in `/status` under each listener entry.
|
||||
OpenMetrics format. Use `/status` for JSON equivalent.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
|
||||
@@ -42,6 +42,37 @@ The Alpine-based image (~59MB) contains Python, PyYAML, and baked-in
|
||||
source. Config is mounted at runtime. The compose.yaml volume mount
|
||||
overrides source for local dev.
|
||||
|
||||
## Systemd Service
|
||||
|
||||
Install s5p as a systemd service for automatic startup and restart.
|
||||
|
||||
```bash
|
||||
# Install the binary
|
||||
cd ~/git/s5p
|
||||
source .venv/bin/activate
|
||||
pip install -e .
|
||||
|
||||
# Copy config
|
||||
sudo mkdir -p /etc/s5p
|
||||
sudo cp config/example.yaml /etc/s5p/s5p.yaml
|
||||
sudo nano /etc/s5p/s5p.yaml # edit with your settings
|
||||
|
||||
# Install the unit (copies service file + daemon-reload)
|
||||
make install-service
|
||||
|
||||
# Enable and start
|
||||
sudo systemctl enable --now s5p
|
||||
|
||||
# Check status
|
||||
sudo systemctl status s5p
|
||||
journalctl -u s5p -f
|
||||
```
|
||||
|
||||
The service unit expects:
|
||||
- Binary at `/usr/local/bin/s5p`
|
||||
- Config at `/etc/s5p/s5p.yaml`
|
||||
- Restarts on failure with 5-second delay
|
||||
|
||||
## Install Tor (optional)
|
||||
|
||||
```bash
|
||||
|
||||
434
docs/USAGE.md
434
docs/USAGE.md
@@ -139,6 +139,26 @@ Each pool has independent health testing, state persistence, and source
|
||||
refresh cycles. The `mitm` source filter adds `?mitm=0` or `?mitm=1` to
|
||||
API requests.
|
||||
|
||||
### Pool protocol filter
|
||||
|
||||
Use `allowed_protos` to restrict a pool to specific proxy protocols.
|
||||
Proxies not matching the list are silently dropped during merge, regardless
|
||||
of source type (API or file).
|
||||
|
||||
```yaml
|
||||
proxy_pools:
|
||||
socks_only:
|
||||
allowed_protos: [socks5] # reject http/socks4 proxies
|
||||
sources:
|
||||
- url: http://api:8081/proxies/all
|
||||
any_proto:
|
||||
sources:
|
||||
- url: http://api:8081/proxies/all # no filter, accept all
|
||||
```
|
||||
|
||||
Valid values: `socks5`, `socks4`, `http`. Visible in `/config` API response
|
||||
when set.
|
||||
|
||||
### Backward compatibility
|
||||
|
||||
The singular `proxy_pool:` key still works -- it registers as pool `"default"`.
|
||||
@@ -523,56 +543,344 @@ api_listen: 127.0.0.1:1081
|
||||
s5p --api 127.0.0.1:1081 -c config/s5p.yaml
|
||||
```
|
||||
|
||||
### Read endpoints
|
||||
Responses are `application/json` unless noted otherwise. Errors return
|
||||
`{"error": "message"}` with appropriate status code (400, 404, 405, 500).
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/status` | Combined summary: uptime, metrics, pool stats, chain |
|
||||
| `GET` | `/metrics` | Full metrics counters (connections, bytes, rate, latency) |
|
||||
| `GET` | `/pool` | All proxies with per-entry state |
|
||||
| `GET` | `/pool/alive` | Alive proxies only |
|
||||
| `GET` | `/config` | Current runtime config (sanitized) |
|
||||
Settings that require a restart: `listen`, `chain`, `pool_size`, `pool_max_idle`, `api_listen`.
|
||||
|
||||
### Write endpoints
|
||||
### API Reference
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `POST` | `/reload` | Re-read config file (replaces SIGHUP) |
|
||||
| `POST` | `/pool/test` | Trigger immediate health test cycle |
|
||||
| `POST` | `/pool/refresh` | Trigger immediate source re-fetch |
|
||||
#### `GET /status`
|
||||
|
||||
All responses are `application/json`. Errors return `{"error": "message"}` with
|
||||
appropriate status code (400, 404, 405, 500).
|
||||
|
||||
### Examples
|
||||
Combined runtime summary: uptime, metrics, pool stats, listeners.
|
||||
|
||||
```bash
|
||||
# Runtime status
|
||||
curl -s http://127.0.0.1:1081/status | jq .
|
||||
```
|
||||
|
||||
# Full metrics
|
||||
curl -s http://127.0.0.1:1081/metrics | jq .
|
||||
```json
|
||||
{
|
||||
"uptime": 3661.2,
|
||||
"connections": 1842,
|
||||
"success": 1790,
|
||||
"failed": 52,
|
||||
"active": 3,
|
||||
"bytes_in": 52428800,
|
||||
"bytes_out": 1073741824,
|
||||
"rate": 4.72,
|
||||
"latency": {"count": 1000, "min": 45.2, "max": 2841.7, "avg": 312.4, "p50": 198.3, "p95": 890.1, "p99": 1523.6},
|
||||
"pool": {"alive": 42, "total": 65},
|
||||
"pools": {
|
||||
"clean": {"alive": 30, "total": 45},
|
||||
"mitm": {"alive": 12, "total": 20}
|
||||
},
|
||||
"tor_nodes": ["socks5://10.200.1.1:9050", "socks5://10.200.1.254:9050"],
|
||||
"listeners": [
|
||||
{
|
||||
"listen": "0.0.0.0:1080",
|
||||
"chain": ["socks5://10.200.1.13:9050"],
|
||||
"pool_hops": 2,
|
||||
"pool": "clean",
|
||||
"auth": true,
|
||||
"latency": {"count": 500, "p50": 1800.2, "p95": 8200.1, "p99": 10500.3, "...": "..."}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
# Pool state (all proxies)
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `uptime` | float | Seconds since server start |
|
||||
| `connections` | int | Total incoming connections |
|
||||
| `success` | int | Successfully relayed |
|
||||
| `failed` | int | All retries exhausted |
|
||||
| `active` | int | Currently relaying |
|
||||
| `bytes_in` | int | Bytes client -> remote |
|
||||
| `bytes_out` | int | Bytes remote -> client |
|
||||
| `rate` | float | Connections/sec (rolling window) |
|
||||
| `latency` | object/null | Aggregate chain setup latency (ms), null if no samples |
|
||||
| `pool` | object | Aggregate pool counts (present when pool active) |
|
||||
| `pools` | object | Per-pool counts (present when multiple pools) |
|
||||
| `tor_nodes` | array | Tor node URLs (present when configured) |
|
||||
| `listeners` | array | Per-listener state with chain, pool, latency |
|
||||
| `listeners[].auth` | bool | Present and `true` when auth is enabled |
|
||||
|
||||
#### `GET /metrics`
|
||||
|
||||
Prometheus/OpenMetrics exposition format. Content-Type:
|
||||
`application/openmetrics-text; version=1.0.0; charset=utf-8`.
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:1081/metrics
|
||||
```
|
||||
|
||||
```
|
||||
# HELP s5p_connections Total connection attempts.
|
||||
# TYPE s5p_connections counter
|
||||
s5p_connections_total 1842
|
||||
# HELP s5p_connections_success Connections successfully relayed.
|
||||
# TYPE s5p_connections_success counter
|
||||
s5p_connections_success_total 1790
|
||||
# HELP s5p_connections_failed Connection failures.
|
||||
# TYPE s5p_connections_failed counter
|
||||
s5p_connections_failed_total 52
|
||||
# HELP s5p_retries Connection retry attempts.
|
||||
# TYPE s5p_retries counter
|
||||
s5p_retries_total 67
|
||||
# HELP s5p_auth_failures SOCKS5 authentication failures.
|
||||
# TYPE s5p_auth_failures counter
|
||||
s5p_auth_failures_total 0
|
||||
# HELP s5p_bytes_in Bytes received from clients.
|
||||
# TYPE s5p_bytes_in counter
|
||||
s5p_bytes_in_total 52428800
|
||||
# HELP s5p_bytes_out Bytes sent to clients.
|
||||
# TYPE s5p_bytes_out counter
|
||||
s5p_bytes_out_total 1073741824
|
||||
# HELP s5p_active_connections Currently open connections.
|
||||
# TYPE s5p_active_connections gauge
|
||||
s5p_active_connections 3
|
||||
# HELP s5p_uptime_seconds Seconds since server start.
|
||||
# TYPE s5p_uptime_seconds gauge
|
||||
s5p_uptime_seconds 3661.2
|
||||
# HELP s5p_connection_rate Connections per second (rolling window).
|
||||
# TYPE s5p_connection_rate gauge
|
||||
s5p_connection_rate 4.72
|
||||
# HELP s5p_pool_proxies_alive Alive proxies in pool.
|
||||
# TYPE s5p_pool_proxies_alive gauge
|
||||
s5p_pool_proxies_alive{pool="clean"} 30
|
||||
s5p_pool_proxies_alive{pool="mitm"} 12
|
||||
# HELP s5p_pool_proxies_total Total proxies in pool.
|
||||
# TYPE s5p_pool_proxies_total gauge
|
||||
s5p_pool_proxies_total{pool="clean"} 45
|
||||
s5p_pool_proxies_total{pool="mitm"} 20
|
||||
# HELP s5p_chain_latency_seconds Chain build latency in seconds.
|
||||
# TYPE s5p_chain_latency_seconds summary
|
||||
s5p_chain_latency_seconds{quantile="0.5"} 0.198300
|
||||
s5p_chain_latency_seconds{quantile="0.95"} 0.890100
|
||||
s5p_chain_latency_seconds{quantile="0.99"} 1.523600
|
||||
s5p_chain_latency_seconds_count 1000
|
||||
s5p_chain_latency_seconds_sum 312.400000
|
||||
# EOF
|
||||
```
|
||||
|
||||
**Metrics exposed:**
|
||||
|
||||
| Metric | Type | Labels | Description |
|
||||
|--------|------|--------|-------------|
|
||||
| `s5p_connections` | counter | -- | Total connection attempts |
|
||||
| `s5p_connections_success` | counter | -- | Successfully relayed |
|
||||
| `s5p_connections_failed` | counter | -- | Connection failures |
|
||||
| `s5p_retries` | counter | -- | Retry attempts |
|
||||
| `s5p_auth_failures` | counter | -- | SOCKS5 auth failures |
|
||||
| `s5p_bytes_in` | counter | -- | Bytes received from clients |
|
||||
| `s5p_bytes_out` | counter | -- | Bytes sent to clients |
|
||||
| `s5p_active_connections` | gauge | -- | Currently open connections |
|
||||
| `s5p_uptime_seconds` | gauge | -- | Seconds since server start |
|
||||
| `s5p_connection_rate` | gauge | -- | Connections/sec (rolling window) |
|
||||
| `s5p_pool_proxies_alive` | gauge | `pool` | Alive proxies per pool |
|
||||
| `s5p_pool_proxies_total` | gauge | `pool` | Total proxies per pool |
|
||||
| `s5p_chain_latency_seconds` | summary | `quantile` | Chain build latency (p50/p95/p99) |
|
||||
| `s5p_listener_chain_latency_seconds` | summary | `listener`, `quantile` | Per-listener chain latency |
|
||||
|
||||
**Prometheus scrape config:**
|
||||
|
||||
```yaml
|
||||
scrape_configs:
|
||||
- job_name: s5p
|
||||
metrics_path: /metrics
|
||||
static_configs:
|
||||
- targets: ["127.0.0.1:1081"]
|
||||
```
|
||||
|
||||
#### `GET /pool`
|
||||
|
||||
All proxies with per-entry state.
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:1081/pool | jq .
|
||||
```
|
||||
|
||||
# Alive proxies only
|
||||
```json
|
||||
{
|
||||
"alive": 42,
|
||||
"total": 65,
|
||||
"pools": {
|
||||
"clean": {"alive": 30, "total": 45},
|
||||
"mitm": {"alive": 12, "total": 20}
|
||||
},
|
||||
"proxies": {
|
||||
"socks5://1.2.3.4:1080": {
|
||||
"alive": true,
|
||||
"fails": 0,
|
||||
"tests": 12,
|
||||
"last_ok": 1708012345.6,
|
||||
"last_test": 1708012345.6,
|
||||
"last_seen": 1708012300.0,
|
||||
"pool": "clean"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `alive` | int | Total alive proxies across all pools |
|
||||
| `total` | int | Total proxies across all pools |
|
||||
| `pools` | object | Per-pool counts (present when multiple pools) |
|
||||
| `proxies` | object | Keyed by proxy URL |
|
||||
| `proxies[].alive` | bool | Currently passing health tests |
|
||||
| `proxies[].fails` | int | Consecutive failures |
|
||||
| `proxies[].tests` | int | Total health tests performed |
|
||||
| `proxies[].last_ok` | float | Unix timestamp of last successful test |
|
||||
| `proxies[].last_test` | float | Unix timestamp of last test (pass or fail) |
|
||||
| `proxies[].last_seen` | float | Unix timestamp of last source refresh that included this proxy |
|
||||
| `proxies[].pool` | string | Pool name (present when multiple pools) |
|
||||
|
||||
#### `GET /pool/alive`
|
||||
|
||||
Same schema as `/pool`, filtered to alive proxies only.
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:1081/pool/alive | jq '.proxies | length'
|
||||
```
|
||||
|
||||
# Current config
|
||||
#### `GET /config`
|
||||
|
||||
Current runtime config (sanitized -- passwords are never exposed).
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:1081/config | jq .
|
||||
```
|
||||
|
||||
# Reload config (like SIGHUP)
|
||||
```json
|
||||
{
|
||||
"timeout": 10,
|
||||
"retries": 3,
|
||||
"log_level": "info",
|
||||
"max_connections": 256,
|
||||
"pool_size": 0,
|
||||
"listeners": [
|
||||
{
|
||||
"listen": "0.0.0.0:1080",
|
||||
"chain": ["socks5://10.200.1.13:9050"],
|
||||
"pool_hops": 2,
|
||||
"pool": "clean",
|
||||
"auth_users": 2
|
||||
}
|
||||
],
|
||||
"tor_nodes": ["socks5://10.200.1.1:9050"],
|
||||
"proxy_pools": {
|
||||
"clean": {
|
||||
"sources": [{"url": "http://10.200.1.250:8081/proxies/all", "mitm": false}],
|
||||
"refresh": 300,
|
||||
"test_interval": 120,
|
||||
"max_fails": 3
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `timeout` | float | Per-hop connection timeout (seconds) |
|
||||
| `retries` | int | Max connection attempts per request |
|
||||
| `log_level` | string | Current log level |
|
||||
| `max_connections` | int | Concurrent connection cap |
|
||||
| `pool_size` | int | Pre-warmed TCP connections to first hop |
|
||||
| `listeners` | array | Listener configs |
|
||||
| `listeners[].auth_users` | int | Number of auth users (present when auth enabled) |
|
||||
| `tor_nodes` | array | Tor node URLs (present when configured) |
|
||||
| `proxy_pools` | object | Pool configs (present when pools configured) |
|
||||
|
||||
#### `GET /tor`
|
||||
|
||||
Tor controller status.
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:1081/tor | jq .
|
||||
```
|
||||
|
||||
```json
|
||||
{"enabled": true, "connected": true, "last_newnym": 45.2, "newnym_interval": 60}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `enabled` | bool | Whether Tor control is configured |
|
||||
| `connected` | bool | Whether connected to Tor control port |
|
||||
| `last_newnym` | float/null | Seconds since last NEWNYM signal |
|
||||
| `newnym_interval` | int | Auto-rotation interval (0 = manual) |
|
||||
|
||||
Returns `{"enabled": false}` when Tor control is not configured.
|
||||
|
||||
#### `POST /reload`
|
||||
|
||||
Re-read config file (equivalent to SIGHUP).
|
||||
|
||||
```bash
|
||||
curl -s -X POST http://127.0.0.1:1081/reload | jq .
|
||||
```
|
||||
|
||||
# Trigger health tests now
|
||||
```json
|
||||
{"ok": true}
|
||||
```
|
||||
|
||||
Returns `{"error": "..."}` (500) on failure.
|
||||
|
||||
#### `POST /pool/test`
|
||||
|
||||
Trigger immediate health test cycle for all pools.
|
||||
|
||||
```bash
|
||||
curl -s -X POST http://127.0.0.1:1081/pool/test | jq .
|
||||
```
|
||||
|
||||
# Re-fetch proxy sources now
|
||||
```json
|
||||
{"ok": true}
|
||||
```
|
||||
|
||||
Returns `{"error": "no proxy pool configured"}` (400) when no pool is active.
|
||||
|
||||
#### `POST /pool/refresh`
|
||||
|
||||
Trigger immediate source re-fetch for all pools.
|
||||
|
||||
```bash
|
||||
curl -s -X POST http://127.0.0.1:1081/pool/refresh | jq .
|
||||
```
|
||||
|
||||
Settings that require a restart: `listen`, `chain`, `pool_size`, `pool_max_idle`, `api_listen`.
|
||||
```json
|
||||
{"ok": true}
|
||||
```
|
||||
|
||||
Returns `{"error": "no proxy pool configured"}` (400) when no pool is active.
|
||||
|
||||
#### `POST /tor/newnym`
|
||||
|
||||
Request new Tor circuit (NEWNYM signal).
|
||||
|
||||
```bash
|
||||
curl -s -X POST http://127.0.0.1:1081/tor/newnym | jq .
|
||||
```
|
||||
|
||||
```json
|
||||
{"ok": true}
|
||||
```
|
||||
|
||||
Returns `{"ok": false, "reason": "rate-limited or not connected"}` when the
|
||||
signal cannot be sent. Returns `{"error": "tor control not configured"}` (400)
|
||||
when Tor control is not configured.
|
||||
|
||||
#### Error responses
|
||||
|
||||
All endpoints return errors as JSON with appropriate HTTP status codes:
|
||||
|
||||
| Status | Meaning | Example |
|
||||
|--------|---------|---------|
|
||||
| 400 | Bad request | `{"error": "no proxy pool configured"}` |
|
||||
| 404 | Unknown path | `{"error": "not found"}` |
|
||||
| 405 | Wrong method | `{"error": "use GET for /status"}` |
|
||||
| 500 | Server error | `{"error": "reload not available"}` |
|
||||
|
||||
## Tor Control Port
|
||||
|
||||
@@ -651,6 +959,31 @@ retries: 5 # try up to 5 different proxies per connection
|
||||
s5p -r 5 -C socks5://127.0.0.1:9050 -S http://api:8081/proxies
|
||||
```
|
||||
|
||||
### Per-listener retry override
|
||||
|
||||
Each listener can override the global `retries` setting. Set `retries` on
|
||||
a listener to use a different retry count for that port. A value of 0 (or
|
||||
omitting the key) inherits the global setting.
|
||||
|
||||
```yaml
|
||||
retries: 3 # global default
|
||||
|
||||
listeners:
|
||||
- listen: 0.0.0.0:1080
|
||||
retries: 5 # deep chain: more retries
|
||||
chain:
|
||||
- socks5://127.0.0.1:9050
|
||||
- pool
|
||||
- pool
|
||||
- listen: 0.0.0.0:1082
|
||||
chain:
|
||||
- socks5://127.0.0.1:9050 # Tor only: uses global retries=3
|
||||
```
|
||||
|
||||
The effective retry count for a listener is `listener.retries` if set,
|
||||
otherwise `config.retries`. Visible in `/config` and `/status` API responses
|
||||
when overridden.
|
||||
|
||||
## Hot Reload
|
||||
|
||||
Send `SIGHUP` to reload the config file without restarting:
|
||||
@@ -700,48 +1033,11 @@ metrics: conn=1842 ok=1790 fail=52 retries=67 active=3 in=50.0M out=1.0G rate=4.
|
||||
| `up` | Server uptime |
|
||||
| `pool` | Alive/total proxies (only when pool is active) |
|
||||
|
||||
### `/metrics` JSON response
|
||||
### `/metrics` OpenMetrics endpoint
|
||||
|
||||
`GET /metrics` returns all counters plus rate, latency percentiles, and
|
||||
per-listener latency breakdowns:
|
||||
|
||||
```json
|
||||
{
|
||||
"connections": 1842,
|
||||
"success": 1790,
|
||||
"failed": 52,
|
||||
"retries": 67,
|
||||
"active": 3,
|
||||
"bytes_in": 52428800,
|
||||
"bytes_out": 1073741824,
|
||||
"uptime": 3661.2,
|
||||
"rate": 4.72,
|
||||
"latency": {
|
||||
"count": 1000,
|
||||
"min": 45.2,
|
||||
"max": 2841.7,
|
||||
"avg": 312.4,
|
||||
"p50": 198.3,
|
||||
"p95": 890.1,
|
||||
"p99": 1523.6
|
||||
},
|
||||
"listener_latency": {
|
||||
"0.0.0.0:1080": {"count": 500, "min": 800.1, "max": 12400.3, "avg": 2100.5, "p50": 1800.2, "p95": 8200.1, "p99": 10500.3},
|
||||
"0.0.0.0:1081": {"count": 300, "min": 400.5, "max": 5200.1, "avg": 1200.3, "p50": 1000.1, "p95": 3500.2, "p99": 4800.7},
|
||||
"0.0.0.0:1082": {"count": 200, "min": 150.2, "max": 2000.1, "avg": 500.3, "p50": 400.1, "p95": 1200.5, "p99": 1800.2}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `rate` | float | Connections/sec (rolling window of last 256 events) |
|
||||
| `latency` | object/null | Aggregate chain setup latency in ms (null if no samples) |
|
||||
| `latency.count` | int | Number of samples in buffer (max 1000) |
|
||||
| `latency.p50` | float | Median latency (ms) |
|
||||
| `latency.p95` | float | 95th percentile (ms) |
|
||||
| `latency.p99` | float | 99th percentile (ms) |
|
||||
| `listener_latency` | object | Per-listener latency, keyed by `host:port` |
|
||||
`GET /metrics` returns all counters, gauges, pool stats, and latency summaries
|
||||
in OpenMetrics format (see [API Reference](#get-metrics) above). Use `/status`
|
||||
for the JSON equivalent with aggregate data.
|
||||
|
||||
### Per-listener latency
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "s5p"
|
||||
version = "0.1.0"
|
||||
version = "0.3.0"
|
||||
description = "SOCKS5 proxy with Tor and proxy-chain support"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = ["pyyaml>=6.0"]
|
||||
|
||||
1
requirements.txt
Normal file
1
requirements.txt
Normal file
@@ -0,0 +1 @@
|
||||
pyyaml>=6.0
|
||||
@@ -1,3 +1,3 @@
|
||||
"""s5p -- SOCKS5 proxy with chain support."""
|
||||
|
||||
__version__ = "0.1.0"
|
||||
__version__ = "0.3.0"
|
||||
|
||||
160
src/s5p/api.py
160
src/s5p/api.py
@@ -30,23 +30,33 @@ def _parse_request(data: bytes) -> tuple[str, str]:
|
||||
return parts[0].upper(), parts[1].split("?", 1)[0]
|
||||
|
||||
|
||||
def _http_response(
|
||||
writer: asyncio.StreamWriter,
|
||||
status: int,
|
||||
payload: bytes,
|
||||
content_type: str = "application/json",
|
||||
) -> None:
|
||||
"""Write an HTTP response and close."""
|
||||
phrases = {200: "OK", 400: "Bad Request", 404: "Not Found",
|
||||
405: "Method Not Allowed", 500: "Internal Server Error"}
|
||||
header = (
|
||||
f"HTTP/1.1 {status} {phrases.get(status, 'Error')}\r\n"
|
||||
f"Content-Type: {content_type}\r\n"
|
||||
f"Content-Length: {len(payload)}\r\n"
|
||||
f"Connection: close\r\n"
|
||||
f"\r\n"
|
||||
)
|
||||
writer.write(header.encode() + payload)
|
||||
|
||||
|
||||
def _json_response(
|
||||
writer: asyncio.StreamWriter,
|
||||
status: int,
|
||||
body: dict | list,
|
||||
) -> None:
|
||||
"""Write an HTTP response with JSON body and close."""
|
||||
phrases = {200: "OK", 400: "Bad Request", 404: "Not Found",
|
||||
405: "Method Not Allowed", 500: "Internal Server Error"}
|
||||
payload = json.dumps(body, separators=(",", ":")).encode()
|
||||
header = (
|
||||
f"HTTP/1.1 {status} {phrases.get(status, 'Error')}\r\n"
|
||||
f"Content-Type: application/json\r\n"
|
||||
f"Content-Length: {len(payload)}\r\n"
|
||||
f"Connection: close\r\n"
|
||||
f"\r\n"
|
||||
)
|
||||
writer.write(header.encode() + payload)
|
||||
_http_response(writer, status, payload, "application/json")
|
||||
|
||||
|
||||
# -- helpers -----------------------------------------------------------------
|
||||
@@ -103,6 +113,7 @@ def _handle_status(ctx: dict) -> tuple[int, dict]:
|
||||
**({"pool": lc.pool_name} if lc.pool_name else {}),
|
||||
**(_pool_seq_entry(lc) if _multi_pool(lc) else {}),
|
||||
**({"auth": True} if lc.auth else {}),
|
||||
**({"retries": lc.retries} if lc.retries else {}),
|
||||
"latency": metrics.get_listener_latency(
|
||||
f"{lc.listen_host}:{lc.listen_port}"
|
||||
).stats(),
|
||||
@@ -112,9 +123,111 @@ def _handle_status(ctx: dict) -> tuple[int, dict]:
|
||||
return 200, data
|
||||
|
||||
|
||||
def _handle_metrics(ctx: dict) -> tuple[int, dict]:
|
||||
"""GET /metrics -- full metrics counters."""
|
||||
return 200, ctx["metrics"].to_dict()
|
||||
def _render_openmetrics(ctx: dict) -> str:
|
||||
"""Render all metrics in OpenMetrics text format."""
|
||||
m: Metrics = ctx["metrics"]
|
||||
lines: list[str] = []
|
||||
|
||||
def _counter(name: str, help_text: str, value: int) -> None:
|
||||
lines.append(f"# HELP {name} {help_text}")
|
||||
lines.append(f"# TYPE {name} counter")
|
||||
lines.append(f"{name}_total {value}")
|
||||
|
||||
def _gauge(name: str, help_text: str, value: float) -> None:
|
||||
lines.append(f"# HELP {name} {help_text}")
|
||||
lines.append(f"# TYPE {name} gauge")
|
||||
lines.append(f"{name} {value}")
|
||||
|
||||
def _summary(name: str, help_text: str, q: dict,
|
||||
labels: str = "") -> None:
|
||||
lines.append(f"# HELP {name} {help_text}")
|
||||
lines.append(f"# TYPE {name} summary")
|
||||
lb = f"{{{labels}," if labels else "{"
|
||||
for quantile in ("0.5", "0.95", "0.99"):
|
||||
lines.append(f'{name}{lb}quantile="{quantile}"}} {q[quantile]:.6f}')
|
||||
lw = f"{{{labels}}}" if labels else ""
|
||||
lines.append(f"{name}_count{lw} {q['count']}")
|
||||
lines.append(f"{name}_sum{lw} {q['sum']:.6f}")
|
||||
|
||||
# -- counters
|
||||
_counter("s5p_connections", "Total connection attempts.", m.connections)
|
||||
_counter("s5p_connections_success",
|
||||
"Connections successfully relayed.", m.success)
|
||||
_counter("s5p_connections_failed", "Connection failures.", m.failed)
|
||||
_counter("s5p_retries", "Connection retry attempts.", m.retries)
|
||||
_counter("s5p_auth_failures",
|
||||
"SOCKS5 authentication failures.", m.auth_failures)
|
||||
_counter("s5p_bytes_in",
|
||||
"Bytes received from clients.", m.bytes_in)
|
||||
_counter("s5p_bytes_out",
|
||||
"Bytes sent to clients.", m.bytes_out)
|
||||
|
||||
# -- gauges
|
||||
_gauge("s5p_active_connections",
|
||||
"Currently open connections.", m.active)
|
||||
_gauge("s5p_uptime_seconds",
|
||||
"Seconds since server start.",
|
||||
round(time.monotonic() - m.started, 1))
|
||||
_gauge("s5p_connection_rate",
|
||||
"Connections per second (rolling window).",
|
||||
round(m.conn_rate.rate(), 4))
|
||||
|
||||
# -- pool gauges
|
||||
pools: dict = ctx.get("pools") or {}
|
||||
if pools:
|
||||
lines.append("# HELP s5p_pool_proxies_alive Alive proxies in pool.")
|
||||
lines.append("# TYPE s5p_pool_proxies_alive gauge")
|
||||
for name, p in pools.items():
|
||||
lines.append(f's5p_pool_proxies_alive{{pool="{name}"}} {p.alive_count}')
|
||||
lines.append("# HELP s5p_pool_proxies_total Total proxies in pool.")
|
||||
lines.append("# TYPE s5p_pool_proxies_total gauge")
|
||||
for name, p in pools.items():
|
||||
lines.append(f's5p_pool_proxies_total{{pool="{name}"}} {p.count}')
|
||||
elif ctx.get("pool"):
|
||||
p = ctx["pool"]
|
||||
_gauge("s5p_pool_proxies_alive", "Alive proxies in pool.", p.alive_count)
|
||||
_gauge("s5p_pool_proxies_total", "Total proxies in pool.", p.count)
|
||||
|
||||
# -- latency summary (global)
|
||||
q = m.latency.quantiles()
|
||||
if q:
|
||||
_summary("s5p_chain_latency_seconds",
|
||||
"Chain build latency in seconds.", q)
|
||||
|
||||
# -- per-listener latency summaries
|
||||
if m.listener_latency:
|
||||
lines.append(
|
||||
"# HELP s5p_listener_chain_latency_seconds "
|
||||
"Per-listener chain build latency in seconds.")
|
||||
lines.append("# TYPE s5p_listener_chain_latency_seconds summary")
|
||||
for key, tracker in m.listener_latency.items():
|
||||
lq = tracker.quantiles()
|
||||
if not lq:
|
||||
continue
|
||||
for quantile in ("0.5", "0.95", "0.99"):
|
||||
lines.append(
|
||||
f's5p_listener_chain_latency_seconds'
|
||||
f'{{listener="{key}",quantile="{quantile}"}} '
|
||||
f'{lq[quantile]:.6f}')
|
||||
lines.append(
|
||||
f's5p_listener_chain_latency_seconds_count'
|
||||
f'{{listener="{key}"}} {lq["count"]}')
|
||||
lines.append(
|
||||
f's5p_listener_chain_latency_seconds_sum'
|
||||
f'{{listener="{key}"}} {lq["sum"]:.6f}')
|
||||
|
||||
lines.append("# EOF")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
_OPENMETRICS_CT = (
|
||||
"application/openmetrics-text; version=1.0.0; charset=utf-8"
|
||||
)
|
||||
|
||||
|
||||
def _handle_metrics(ctx: dict) -> tuple[int, str]:
|
||||
"""GET /metrics -- OpenMetrics exposition."""
|
||||
return 200, _render_openmetrics(ctx)
|
||||
|
||||
|
||||
def _handle_pool(ctx: dict, alive_only: bool = False) -> tuple[int, dict]:
|
||||
@@ -183,6 +296,7 @@ def _handle_config(ctx: dict) -> tuple[int, dict]:
|
||||
**({"pool": lc.pool_name} if lc.pool_name else {}),
|
||||
**(_pool_seq_entry(lc) if _multi_pool(lc) else {}),
|
||||
**({"auth_users": len(lc.auth)} if lc.auth else {}),
|
||||
**({"retries": lc.retries} if lc.retries else {}),
|
||||
}
|
||||
for lc in config.listeners
|
||||
],
|
||||
@@ -202,12 +316,15 @@ def _handle_config(ctx: dict) -> tuple[int, dict]:
|
||||
if src.mitm is not None:
|
||||
s["mitm"] = src.mitm
|
||||
sources.append(s)
|
||||
pools_data[name] = {
|
||||
pool_entry: dict = {
|
||||
"sources": sources,
|
||||
"refresh": pp.refresh,
|
||||
"test_interval": pp.test_interval,
|
||||
"max_fails": pp.max_fails,
|
||||
}
|
||||
if pp.allowed_protos:
|
||||
pool_entry["allowed_protos"] = pp.allowed_protos
|
||||
pools_data[name] = pool_entry
|
||||
data["proxy_pools"] = pools_data
|
||||
elif config.proxy_pool:
|
||||
pp = config.proxy_pool
|
||||
@@ -310,8 +427,13 @@ _POST_ROUTES: dict[str, str] = {
|
||||
}
|
||||
|
||||
|
||||
async def _route(method: str, path: str, ctx: dict) -> tuple[int, dict]:
|
||||
"""Dispatch request to the appropriate handler."""
|
||||
async def _route(
|
||||
method: str, path: str, ctx: dict,
|
||||
) -> tuple[int, dict | str]:
|
||||
"""Dispatch request to the appropriate handler.
|
||||
|
||||
Returns (status, body) where body is a dict (JSON) or str (text).
|
||||
"""
|
||||
if method == "GET" and path in _GET_ROUTES:
|
||||
name = _GET_ROUTES[path]
|
||||
if name == "status":
|
||||
@@ -366,7 +488,11 @@ async def _handle_connection(
|
||||
return
|
||||
|
||||
status, body = await _route(method, path, ctx)
|
||||
_json_response(writer, status, body)
|
||||
if isinstance(body, str):
|
||||
_http_response(writer, status, body.encode(),
|
||||
_OPENMETRICS_CT)
|
||||
else:
|
||||
_json_response(writer, status, body)
|
||||
await writer.drain()
|
||||
except (TimeoutError, ConnectionError, OSError):
|
||||
pass
|
||||
|
||||
@@ -57,6 +57,7 @@ class ProxyPoolConfig:
|
||||
max_fails: int = 3
|
||||
state_file: str = ""
|
||||
report_url: str = ""
|
||||
allowed_protos: list[str] = field(default_factory=list)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Backward compat: extract hostname from legacy test_url."""
|
||||
@@ -89,6 +90,7 @@ class ListenerConfig:
|
||||
pool_name: str = ""
|
||||
bypass: list[str] = field(default_factory=list)
|
||||
auth: dict[str, str] = field(default_factory=dict)
|
||||
retries: int = 0 # 0 = use global default
|
||||
|
||||
@property
|
||||
def pool_hops(self) -> int:
|
||||
@@ -192,6 +194,8 @@ def _parse_pool_config(pool_raw: dict) -> ProxyPoolConfig:
|
||||
}
|
||||
if "test_targets" in pool_raw:
|
||||
kwargs["test_targets"] = list(pool_raw["test_targets"])
|
||||
if "allowed_protos" in pool_raw:
|
||||
kwargs["allowed_protos"] = list(pool_raw["allowed_protos"])
|
||||
return ProxyPoolConfig(**kwargs)
|
||||
|
||||
|
||||
@@ -330,6 +334,8 @@ def load_config(path: str | Path) -> Config:
|
||||
auth_raw = entry["auth"]
|
||||
if isinstance(auth_raw, dict):
|
||||
lc.auth = {str(k): str(v) for k, v in auth_raw.items()}
|
||||
if "retries" in entry:
|
||||
lc.retries = int(entry["retries"])
|
||||
if "pool" in entry:
|
||||
lc.pool_name = entry["pool"]
|
||||
default_pool = lc.pool_name or "default"
|
||||
|
||||
@@ -67,6 +67,23 @@ class LatencyTracker:
|
||||
"p99": round(ms[min(int(n * 0.99), n - 1)], 1),
|
||||
}
|
||||
|
||||
def quantiles(self) -> dict | None:
|
||||
"""Return ``{count, sum, 0.5, 0.95, 0.99}`` in seconds, or None.
|
||||
|
||||
Designed for Prometheus/OpenMetrics summary exposition.
|
||||
"""
|
||||
n = len(self._samples)
|
||||
if n == 0:
|
||||
return None
|
||||
s = sorted(self._samples)
|
||||
return {
|
||||
"count": n,
|
||||
"sum": sum(s),
|
||||
"0.5": s[int(n * 0.50)],
|
||||
"0.95": s[min(int(n * 0.95), n - 1)],
|
||||
"0.99": s[min(int(n * 0.99), n - 1)],
|
||||
}
|
||||
|
||||
|
||||
class Metrics:
|
||||
"""Connection metrics.
|
||||
|
||||
@@ -241,6 +241,8 @@ class ProxyPool:
|
||||
now = time.time()
|
||||
seen: set[str] = set()
|
||||
for hop in proxies:
|
||||
if self._cfg.allowed_protos and hop.proto not in self._cfg.allowed_protos:
|
||||
continue
|
||||
key = f"{hop.proto}://{hop.host}:{hop.port}"
|
||||
seen.add(key)
|
||||
if key in self._proxies:
|
||||
|
||||
@@ -208,6 +208,7 @@ async def _handle_client(
|
||||
else:
|
||||
effective_chain = list(listener.chain)
|
||||
fhp = first_hop_pool
|
||||
# tor_nodes round-robin overrides the listener's first hop
|
||||
if tor_rr and effective_chain:
|
||||
node = tor_rr.next()
|
||||
effective_chain[0] = node
|
||||
@@ -458,7 +459,8 @@ async def serve(config: Config) -> None:
|
||||
) -> None:
|
||||
async with sem:
|
||||
await _handle_client(
|
||||
r, w, _lc, config.timeout, config.retries,
|
||||
r, w, _lc, config.timeout,
|
||||
_lc.retries or config.retries,
|
||||
_pools, metrics, _hp, tor_rr, hop_pools,
|
||||
)
|
||||
|
||||
|
||||
@@ -13,6 +13,7 @@ from s5p.api import (
|
||||
_handle_tor_newnym,
|
||||
_json_response,
|
||||
_parse_request,
|
||||
_render_openmetrics,
|
||||
_route,
|
||||
)
|
||||
from s5p.config import ChainHop, Config, ListenerConfig, PoolSourceConfig, ProxyPoolConfig
|
||||
@@ -222,6 +223,87 @@ class TestHandleConfigAuth:
|
||||
assert "s3cret" not in str(body)
|
||||
|
||||
|
||||
class TestHandleStatusRetries:
|
||||
"""Test retries in /status listener entries."""
|
||||
|
||||
def test_retries_present_when_set(self):
|
||||
config = Config(
|
||||
listeners=[
|
||||
ListenerConfig(
|
||||
listen_host="0.0.0.0", listen_port=1080,
|
||||
retries=5,
|
||||
),
|
||||
],
|
||||
)
|
||||
ctx = _make_ctx(config=config)
|
||||
_, body = _handle_status(ctx)
|
||||
assert body["listeners"][0]["retries"] == 5
|
||||
|
||||
def test_retries_absent_when_zero(self):
|
||||
config = Config(
|
||||
listeners=[
|
||||
ListenerConfig(listen_host="0.0.0.0", listen_port=1080),
|
||||
],
|
||||
)
|
||||
ctx = _make_ctx(config=config)
|
||||
_, body = _handle_status(ctx)
|
||||
assert "retries" not in body["listeners"][0]
|
||||
|
||||
|
||||
class TestHandleConfigRetries:
|
||||
"""Test retries in /config listener entries."""
|
||||
|
||||
def test_retries_present_when_set(self):
|
||||
config = Config(
|
||||
listeners=[
|
||||
ListenerConfig(
|
||||
listen_host="0.0.0.0", listen_port=1080,
|
||||
retries=7,
|
||||
),
|
||||
],
|
||||
)
|
||||
ctx = _make_ctx(config=config)
|
||||
_, body = _handle_config(ctx)
|
||||
assert body["listeners"][0]["retries"] == 7
|
||||
|
||||
def test_retries_absent_when_zero(self):
|
||||
config = Config(
|
||||
listeners=[
|
||||
ListenerConfig(listen_host="0.0.0.0", listen_port=1080),
|
||||
],
|
||||
)
|
||||
ctx = _make_ctx(config=config)
|
||||
_, body = _handle_config(ctx)
|
||||
assert "retries" not in body["listeners"][0]
|
||||
|
||||
|
||||
class TestHandleConfigAllowedProtos:
|
||||
"""Test allowed_protos in /config pool entries."""
|
||||
|
||||
def test_allowed_protos_present(self):
|
||||
pp = ProxyPoolConfig(
|
||||
sources=[],
|
||||
allowed_protos=["socks5"],
|
||||
)
|
||||
config = Config(
|
||||
proxy_pools={"socks_only": pp},
|
||||
listeners=[ListenerConfig()],
|
||||
)
|
||||
ctx = _make_ctx(config=config)
|
||||
_, body = _handle_config(ctx)
|
||||
assert body["proxy_pools"]["socks_only"]["allowed_protos"] == ["socks5"]
|
||||
|
||||
def test_allowed_protos_absent_when_empty(self):
|
||||
pp = ProxyPoolConfig(sources=[])
|
||||
config = Config(
|
||||
proxy_pools={"default": pp},
|
||||
listeners=[ListenerConfig()],
|
||||
)
|
||||
ctx = _make_ctx(config=config)
|
||||
_, body = _handle_config(ctx)
|
||||
assert "allowed_protos" not in body["proxy_pools"]["default"]
|
||||
|
||||
|
||||
class TestHandleStatusPools:
|
||||
"""Test GET /status with multiple named pools."""
|
||||
|
||||
@@ -292,20 +374,136 @@ class TestHandleStatusMultiPool:
|
||||
|
||||
|
||||
class TestHandleMetrics:
|
||||
"""Test GET /metrics handler."""
|
||||
"""Test GET /metrics handler (OpenMetrics format)."""
|
||||
|
||||
def test_returns_dict(self):
|
||||
def test_returns_openmetrics_string(self):
|
||||
ctx = _make_ctx()
|
||||
ctx["metrics"].connections = 42
|
||||
ctx["metrics"].bytes_in = 1024
|
||||
status, body = _handle_metrics(ctx)
|
||||
assert status == 200
|
||||
assert body["connections"] == 42
|
||||
assert body["bytes_in"] == 1024
|
||||
assert "uptime" in body
|
||||
assert "rate" in body
|
||||
assert "latency" in body
|
||||
assert "listener_latency" in body
|
||||
assert isinstance(body, str)
|
||||
assert body.rstrip().endswith("# EOF")
|
||||
|
||||
def test_counter_values(self):
|
||||
ctx = _make_ctx()
|
||||
ctx["metrics"].connections = 42
|
||||
ctx["metrics"].bytes_in = 1024
|
||||
_, body = _handle_metrics(ctx)
|
||||
assert "s5p_connections_total 42" in body
|
||||
assert "s5p_bytes_in_total 1024" in body
|
||||
|
||||
|
||||
class TestRenderOpenMetrics:
|
||||
"""Test OpenMetrics text rendering."""
|
||||
|
||||
def test_eof_terminator(self):
|
||||
ctx = _make_ctx()
|
||||
text = _render_openmetrics(ctx)
|
||||
assert text.rstrip().endswith("# EOF")
|
||||
assert text.endswith("\n")
|
||||
|
||||
def test_type_declarations(self):
|
||||
ctx = _make_ctx()
|
||||
text = _render_openmetrics(ctx)
|
||||
assert "# TYPE s5p_connections counter" in text
|
||||
assert "# TYPE s5p_active_connections gauge" in text
|
||||
assert "# TYPE s5p_uptime_seconds gauge" in text
|
||||
|
||||
def test_help_lines(self):
|
||||
ctx = _make_ctx()
|
||||
text = _render_openmetrics(ctx)
|
||||
assert "# HELP s5p_connections Total connection attempts." in text
|
||||
assert "# HELP s5p_active_connections Currently open connections." in text
|
||||
|
||||
def test_counter_values(self):
|
||||
ctx = _make_ctx()
|
||||
ctx["metrics"].connections = 100
|
||||
ctx["metrics"].success = 95
|
||||
ctx["metrics"].failed = 5
|
||||
ctx["metrics"].retries = 10
|
||||
ctx["metrics"].auth_failures = 2
|
||||
ctx["metrics"].bytes_in = 4096
|
||||
ctx["metrics"].bytes_out = 8192
|
||||
text = _render_openmetrics(ctx)
|
||||
assert "s5p_connections_total 100" in text
|
||||
assert "s5p_connections_success_total 95" in text
|
||||
assert "s5p_connections_failed_total 5" in text
|
||||
assert "s5p_retries_total 10" in text
|
||||
assert "s5p_auth_failures_total 2" in text
|
||||
assert "s5p_bytes_in_total 4096" in text
|
||||
assert "s5p_bytes_out_total 8192" in text
|
||||
|
||||
def test_gauge_values(self):
|
||||
ctx = _make_ctx()
|
||||
ctx["metrics"].active = 7
|
||||
text = _render_openmetrics(ctx)
|
||||
assert "s5p_active_connections 7" in text
|
||||
assert "s5p_uptime_seconds " in text
|
||||
assert "s5p_connection_rate " in text
|
||||
|
||||
def test_no_latency_when_empty(self):
|
||||
ctx = _make_ctx()
|
||||
text = _render_openmetrics(ctx)
|
||||
assert "s5p_chain_latency_seconds" not in text
|
||||
|
||||
def test_latency_summary(self):
|
||||
ctx = _make_ctx()
|
||||
for i in range(1, 101):
|
||||
ctx["metrics"].latency.record(i / 1000)
|
||||
text = _render_openmetrics(ctx)
|
||||
assert "# TYPE s5p_chain_latency_seconds summary" in text
|
||||
assert 's5p_chain_latency_seconds{quantile="0.5"}' in text
|
||||
assert 's5p_chain_latency_seconds{quantile="0.95"}' in text
|
||||
assert 's5p_chain_latency_seconds{quantile="0.99"}' in text
|
||||
assert "s5p_chain_latency_seconds_count 100" in text
|
||||
assert "s5p_chain_latency_seconds_sum " in text
|
||||
|
||||
def test_listener_latency_summary(self):
|
||||
ctx = _make_ctx()
|
||||
tracker = ctx["metrics"].get_listener_latency("0.0.0.0:1080")
|
||||
for i in range(1, 51):
|
||||
tracker.record(i / 1000)
|
||||
text = _render_openmetrics(ctx)
|
||||
assert "# TYPE s5p_listener_chain_latency_seconds summary" in text
|
||||
assert (
|
||||
's5p_listener_chain_latency_seconds{listener="0.0.0.0:1080",'
|
||||
'quantile="0.5"}'
|
||||
) in text
|
||||
assert (
|
||||
's5p_listener_chain_latency_seconds_count{listener="0.0.0.0:1080"} 50'
|
||||
) in text
|
||||
|
||||
def test_pool_gauges_multi(self):
|
||||
pool_a = MagicMock()
|
||||
pool_a.alive_count = 5
|
||||
pool_a.count = 10
|
||||
pool_a.name = "clean"
|
||||
pool_b = MagicMock()
|
||||
pool_b.alive_count = 3
|
||||
pool_b.count = 8
|
||||
pool_b.name = "mitm"
|
||||
ctx = _make_ctx(pools={"clean": pool_a, "mitm": pool_b})
|
||||
text = _render_openmetrics(ctx)
|
||||
assert '# TYPE s5p_pool_proxies_alive gauge' in text
|
||||
assert 's5p_pool_proxies_alive{pool="clean"} 5' in text
|
||||
assert 's5p_pool_proxies_alive{pool="mitm"} 3' in text
|
||||
assert 's5p_pool_proxies_total{pool="clean"} 10' in text
|
||||
assert 's5p_pool_proxies_total{pool="mitm"} 8' in text
|
||||
|
||||
def test_pool_gauges_single(self):
|
||||
pool = MagicMock()
|
||||
pool.alive_count = 12
|
||||
pool.count = 20
|
||||
ctx = _make_ctx(pool=pool)
|
||||
text = _render_openmetrics(ctx)
|
||||
assert "s5p_pool_proxies_alive 12" in text
|
||||
assert "s5p_pool_proxies_total 20" in text
|
||||
|
||||
def test_no_pool_metrics_when_unconfigured(self):
|
||||
ctx = _make_ctx()
|
||||
text = _render_openmetrics(ctx)
|
||||
assert "s5p_pool_proxies" not in text
|
||||
|
||||
|
||||
class TestHandlePool:
|
||||
|
||||
153
tests/test_cli.py
Normal file
153
tests/test_cli.py
Normal file
@@ -0,0 +1,153 @@
|
||||
"""Tests for CLI argument parsing."""
|
||||
|
||||
import pytest
|
||||
|
||||
from s5p import __version__
|
||||
from s5p.cli import _parse_args
|
||||
|
||||
|
||||
class TestDefaults:
|
||||
"""Default argument values."""
|
||||
|
||||
def test_no_args(self):
|
||||
args = _parse_args([])
|
||||
assert args.config is None
|
||||
assert args.listen is None
|
||||
assert args.chain is None
|
||||
assert args.timeout is None
|
||||
assert args.retries is None
|
||||
assert args.max_connections is None
|
||||
assert args.verbose is False
|
||||
assert args.quiet is False
|
||||
assert args.proxy_source is None
|
||||
assert args.api is None
|
||||
assert args.cprofile is None
|
||||
assert args.tracemalloc is None
|
||||
|
||||
|
||||
class TestFlags:
|
||||
"""Flag parsing."""
|
||||
|
||||
def test_verbose(self):
|
||||
args = _parse_args(["-v"])
|
||||
assert args.verbose is True
|
||||
|
||||
def test_quiet(self):
|
||||
args = _parse_args(["-q"])
|
||||
assert args.quiet is True
|
||||
|
||||
def test_config(self):
|
||||
args = _parse_args(["-c", "s5p.yaml"])
|
||||
assert args.config == "s5p.yaml"
|
||||
|
||||
def test_config_long(self):
|
||||
args = _parse_args(["--config", "s5p.yaml"])
|
||||
assert args.config == "s5p.yaml"
|
||||
|
||||
def test_listen(self):
|
||||
args = _parse_args(["-l", "0.0.0.0:9999"])
|
||||
assert args.listen == "0.0.0.0:9999"
|
||||
|
||||
def test_chain(self):
|
||||
args = _parse_args(["-C", "socks5://127.0.0.1:9050"])
|
||||
assert args.chain == "socks5://127.0.0.1:9050"
|
||||
|
||||
def test_chain_multi(self):
|
||||
args = _parse_args(["-C", "socks5://a:1080,http://b:8080"])
|
||||
assert args.chain == "socks5://a:1080,http://b:8080"
|
||||
|
||||
def test_timeout(self):
|
||||
args = _parse_args(["-t", "30"])
|
||||
assert args.timeout == 30.0
|
||||
|
||||
def test_retries(self):
|
||||
args = _parse_args(["-r", "5"])
|
||||
assert args.retries == 5
|
||||
|
||||
def test_max_connections(self):
|
||||
args = _parse_args(["-m", "512"])
|
||||
assert args.max_connections == 512
|
||||
|
||||
def test_proxy_source(self):
|
||||
args = _parse_args(["-S", "http://api:8081/proxies"])
|
||||
assert args.proxy_source == "http://api:8081/proxies"
|
||||
|
||||
def test_api(self):
|
||||
args = _parse_args(["--api", "127.0.0.1:1081"])
|
||||
assert args.api == "127.0.0.1:1081"
|
||||
|
||||
def test_cprofile_default(self):
|
||||
args = _parse_args(["--cprofile"])
|
||||
assert args.cprofile == "s5p.prof"
|
||||
|
||||
def test_cprofile_custom(self):
|
||||
args = _parse_args(["--cprofile", "out.prof"])
|
||||
assert args.cprofile == "out.prof"
|
||||
|
||||
def test_tracemalloc_default(self):
|
||||
args = _parse_args(["--tracemalloc"])
|
||||
assert args.tracemalloc == 10
|
||||
|
||||
def test_tracemalloc_custom(self):
|
||||
args = _parse_args(["--tracemalloc", "20"])
|
||||
assert args.tracemalloc == 20
|
||||
|
||||
|
||||
class TestVersion:
|
||||
"""--version flag."""
|
||||
|
||||
def test_version_output(self, capsys):
|
||||
with pytest.raises(SystemExit, match="0"):
|
||||
_parse_args(["--version"])
|
||||
captured = capsys.readouterr()
|
||||
assert captured.out.strip() == f"s5p {__version__}"
|
||||
|
||||
def test_version_short(self, capsys):
|
||||
with pytest.raises(SystemExit, match="0"):
|
||||
_parse_args(["-V"])
|
||||
captured = capsys.readouterr()
|
||||
assert "0.3.0" in captured.out
|
||||
|
||||
|
||||
class TestCombinations:
|
||||
"""Multiple flags together."""
|
||||
|
||||
def test_verbose_with_chain(self):
|
||||
args = _parse_args(["-v", "-C", "socks5://tor:9050"])
|
||||
assert args.verbose is True
|
||||
assert args.chain == "socks5://tor:9050"
|
||||
|
||||
def test_config_with_api(self):
|
||||
args = _parse_args(["-c", "s5p.yaml", "--api", "0.0.0.0:1090"])
|
||||
assert args.config == "s5p.yaml"
|
||||
assert args.api == "0.0.0.0:1090"
|
||||
|
||||
def test_listen_with_timeout_and_retries(self):
|
||||
args = _parse_args(["-l", ":8080", "-t", "15", "-r", "3"])
|
||||
assert args.listen == ":8080"
|
||||
assert args.timeout == 15.0
|
||||
assert args.retries == 3
|
||||
|
||||
|
||||
class TestInvalid:
|
||||
"""Invalid argument handling."""
|
||||
|
||||
def test_unknown_flag(self):
|
||||
with pytest.raises(SystemExit, match="2"):
|
||||
_parse_args(["--nonexistent"])
|
||||
|
||||
def test_timeout_non_numeric(self):
|
||||
with pytest.raises(SystemExit, match="2"):
|
||||
_parse_args(["-t", "abc"])
|
||||
|
||||
def test_retries_non_numeric(self):
|
||||
with pytest.raises(SystemExit, match="2"):
|
||||
_parse_args(["-r", "abc"])
|
||||
|
||||
def test_max_connections_non_numeric(self):
|
||||
with pytest.raises(SystemExit, match="2"):
|
||||
_parse_args(["-m", "abc"])
|
||||
|
||||
def test_tracemalloc_non_numeric(self):
|
||||
with pytest.raises(SystemExit, match="2"):
|
||||
_parse_args(["--tracemalloc", "abc"])
|
||||
@@ -307,6 +307,39 @@ class TestProxyPools:
|
||||
assert c.listeners[1].pool_hops == 0
|
||||
|
||||
|
||||
class TestAllowedProtos:
|
||||
"""Test pool-level allowed_protos config."""
|
||||
|
||||
def test_allowed_protos_from_yaml(self, tmp_path):
|
||||
cfg_file = tmp_path / "test.yaml"
|
||||
cfg_file.write_text(
|
||||
"proxy_pools:\n"
|
||||
" socks_only:\n"
|
||||
" sources: []\n"
|
||||
" allowed_protos: [socks5]\n"
|
||||
" any:\n"
|
||||
" sources: []\n"
|
||||
)
|
||||
c = load_config(cfg_file)
|
||||
assert c.proxy_pools["socks_only"].allowed_protos == ["socks5"]
|
||||
assert c.proxy_pools["any"].allowed_protos == []
|
||||
|
||||
def test_allowed_protos_multiple(self, tmp_path):
|
||||
cfg_file = tmp_path / "test.yaml"
|
||||
cfg_file.write_text(
|
||||
"proxy_pool:\n"
|
||||
" sources: []\n"
|
||||
" allowed_protos: [socks5, http]\n"
|
||||
)
|
||||
c = load_config(cfg_file)
|
||||
assert c.proxy_pool.allowed_protos == ["socks5", "http"]
|
||||
|
||||
def test_allowed_protos_default_empty(self):
|
||||
from s5p.config import ProxyPoolConfig
|
||||
cfg = ProxyPoolConfig()
|
||||
assert cfg.allowed_protos == []
|
||||
|
||||
|
||||
class TestTorNodes:
|
||||
"""Test tor_nodes config parsing."""
|
||||
|
||||
@@ -593,6 +626,40 @@ class TestListenerPoolCompat:
|
||||
assert lc.pool_hops == 0
|
||||
|
||||
|
||||
class TestListenerRetries:
|
||||
"""Test per-listener retry override config."""
|
||||
|
||||
def test_retries_default(self):
|
||||
lc = ListenerConfig()
|
||||
assert lc.retries == 0
|
||||
|
||||
def test_retries_from_yaml(self, tmp_path):
|
||||
cfg_file = tmp_path / "test.yaml"
|
||||
cfg_file.write_text(
|
||||
"listeners:\n"
|
||||
" - listen: 1080\n"
|
||||
" retries: 5\n"
|
||||
" chain:\n"
|
||||
" - socks5://127.0.0.1:9050\n"
|
||||
" - pool\n"
|
||||
" - listen: 1081\n"
|
||||
" chain:\n"
|
||||
" - socks5://127.0.0.1:9050\n"
|
||||
)
|
||||
c = load_config(cfg_file)
|
||||
assert c.listeners[0].retries == 5
|
||||
assert c.listeners[1].retries == 0
|
||||
|
||||
def test_retries_absent_from_yaml(self, tmp_path):
|
||||
cfg_file = tmp_path / "test.yaml"
|
||||
cfg_file.write_text(
|
||||
"listeners:\n"
|
||||
" - listen: 1080\n"
|
||||
)
|
||||
c = load_config(cfg_file)
|
||||
assert c.listeners[0].retries == 0
|
||||
|
||||
|
||||
class TestAuthConfig:
|
||||
"""Test auth field in listener config."""
|
||||
|
||||
|
||||
@@ -52,6 +52,23 @@ class TestLatencyTracker:
|
||||
assert s["min"] == 500.0
|
||||
assert s["max"] == 500.0
|
||||
|
||||
def test_quantiles_empty(self):
|
||||
lt = LatencyTracker()
|
||||
assert lt.quantiles() is None
|
||||
|
||||
def test_quantiles_seconds(self):
|
||||
lt = LatencyTracker()
|
||||
for i in range(1, 101):
|
||||
lt.record(i / 1000)
|
||||
q = lt.quantiles()
|
||||
assert q is not None
|
||||
assert q["count"] == 100
|
||||
assert 0.050 <= q["0.5"] <= 0.052
|
||||
assert 0.095 <= q["0.95"] <= 0.097
|
||||
assert 0.099 <= q["0.99"] <= 0.101
|
||||
assert "sum" in q
|
||||
assert q["sum"] > 0
|
||||
|
||||
|
||||
# -- RateTracker -------------------------------------------------------------
|
||||
|
||||
|
||||
@@ -171,6 +171,46 @@ class TestProxyPoolMerge:
|
||||
assert pool.count == 1
|
||||
|
||||
|
||||
class TestProxyPoolAllowedProtos:
|
||||
"""Test pool-level proxy protocol filter."""
|
||||
|
||||
def test_allowed_protos_filters_merge(self):
|
||||
cfg = ProxyPoolConfig(sources=[], allowed_protos=["socks5"])
|
||||
pool = ProxyPool(cfg, [], timeout=10.0)
|
||||
proxies = [
|
||||
ChainHop(proto="socks5", host="1.2.3.4", port=1080),
|
||||
ChainHop(proto="http", host="5.6.7.8", port=8080),
|
||||
ChainHop(proto="socks4", host="9.9.9.9", port=1080),
|
||||
]
|
||||
pool._merge(proxies)
|
||||
assert pool.count == 1
|
||||
assert "socks5://1.2.3.4:1080" in pool._proxies
|
||||
|
||||
def test_allowed_protos_multiple(self):
|
||||
cfg = ProxyPoolConfig(sources=[], allowed_protos=["socks5", "http"])
|
||||
pool = ProxyPool(cfg, [], timeout=10.0)
|
||||
proxies = [
|
||||
ChainHop(proto="socks5", host="1.2.3.4", port=1080),
|
||||
ChainHop(proto="http", host="5.6.7.8", port=8080),
|
||||
ChainHop(proto="socks4", host="9.9.9.9", port=1080),
|
||||
]
|
||||
pool._merge(proxies)
|
||||
assert pool.count == 2
|
||||
assert "socks5://1.2.3.4:1080" in pool._proxies
|
||||
assert "http://5.6.7.8:8080" in pool._proxies
|
||||
assert "socks4://9.9.9.9:1080" not in pool._proxies
|
||||
|
||||
def test_empty_allowed_protos_accepts_all(self):
|
||||
cfg = ProxyPoolConfig(sources=[], allowed_protos=[])
|
||||
pool = ProxyPool(cfg, [], timeout=10.0)
|
||||
proxies = [
|
||||
ChainHop(proto="socks5", host="1.2.3.4", port=1080),
|
||||
ChainHop(proto="http", host="5.6.7.8", port=8080),
|
||||
]
|
||||
pool._merge(proxies)
|
||||
assert pool.count == 2
|
||||
|
||||
|
||||
class TestProxyPoolGet:
|
||||
"""Test proxy selection."""
|
||||
|
||||
|
||||
@@ -1,6 +1,66 @@
|
||||
"""Tests for protocol helpers."""
|
||||
|
||||
from s5p.proto import Socks5AddrType, encode_address
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from s5p.proto import (
|
||||
ProtoError,
|
||||
Socks5AddrType,
|
||||
Socks5Reply,
|
||||
encode_address,
|
||||
http_connect,
|
||||
socks4_connect,
|
||||
socks5_connect,
|
||||
)
|
||||
|
||||
# -- helpers -----------------------------------------------------------------
|
||||
|
||||
|
||||
class _MockTransport(asyncio.Transport):
|
||||
"""Minimal transport that captures writes and supports drain."""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.written = bytearray()
|
||||
self._closing = False
|
||||
|
||||
def write(self, data):
|
||||
self.written.extend(data)
|
||||
|
||||
def is_closing(self):
|
||||
return self._closing
|
||||
|
||||
def close(self):
|
||||
self._closing = True
|
||||
|
||||
def get_extra_info(self, name, default=None):
|
||||
return default
|
||||
|
||||
|
||||
def _make_streams(response_data: bytes):
|
||||
"""Create mock reader/writer for protocol tests.
|
||||
|
||||
Must be called from within a running event loop.
|
||||
"""
|
||||
reader = asyncio.StreamReader()
|
||||
reader.feed_data(response_data)
|
||||
reader.feed_eof()
|
||||
|
||||
protocol = asyncio.StreamReaderProtocol(reader)
|
||||
transport = _MockTransport()
|
||||
protocol.connection_made(transport)
|
||||
|
||||
writer = asyncio.StreamWriter(transport, protocol, reader, asyncio.get_running_loop())
|
||||
return reader, writer
|
||||
|
||||
|
||||
def _run(coro):
|
||||
"""Run a coroutine in a fresh event loop."""
|
||||
asyncio.run(coro)
|
||||
|
||||
|
||||
# -- encode_address ----------------------------------------------------------
|
||||
|
||||
|
||||
class TestEncodeAddress:
|
||||
@@ -11,12 +71,265 @@ class TestEncodeAddress:
|
||||
assert atyp == Socks5AddrType.IPV4
|
||||
assert data == b"\x7f\x00\x00\x01"
|
||||
|
||||
def test_ipv4_zeros(self):
|
||||
atyp, data = encode_address("0.0.0.0")
|
||||
assert atyp == Socks5AddrType.IPV4
|
||||
assert data == b"\x00\x00\x00\x00"
|
||||
|
||||
def test_ipv6(self):
|
||||
atyp, data = encode_address("::1")
|
||||
assert atyp == Socks5AddrType.IPV6
|
||||
assert len(data) == 16
|
||||
assert data[-1] == 1
|
||||
|
||||
def test_ipv6_full(self):
|
||||
atyp, data = encode_address("2001:db8::1")
|
||||
assert atyp == Socks5AddrType.IPV6
|
||||
assert len(data) == 16
|
||||
|
||||
def test_domain(self):
|
||||
atyp, data = encode_address("example.com")
|
||||
assert atyp == Socks5AddrType.DOMAIN
|
||||
assert data == bytes([11]) + b"example.com"
|
||||
|
||||
def test_domain_short(self):
|
||||
atyp, data = encode_address("a.co")
|
||||
assert atyp == Socks5AddrType.DOMAIN
|
||||
assert data == bytes([4]) + b"a.co"
|
||||
|
||||
def test_domain_long(self):
|
||||
host = "sub.domain.example.com"
|
||||
atyp, data = encode_address(host)
|
||||
assert atyp == Socks5AddrType.DOMAIN
|
||||
assert data[0] == len(host)
|
||||
assert data[1:] == host.encode("ascii")
|
||||
|
||||
|
||||
# -- socks5_connect ----------------------------------------------------------
|
||||
|
||||
|
||||
class TestSocks5Connect:
|
||||
"""Test SOCKS5 handshake building."""
|
||||
|
||||
def test_no_auth_success(self):
|
||||
"""Successful SOCKS5 connect without auth."""
|
||||
bind_addr = b"\x01\x00\x00\x00\x00\x00\x00" # IPv4 0.0.0.0:0
|
||||
response = b"\x05\x00" + b"\x05\x00\x00" + bind_addr
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
await socks5_connect(reader, writer, "example.com", 80)
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_auth_success(self):
|
||||
"""Successful SOCKS5 connect with username/password auth."""
|
||||
bind_addr = b"\x01\x00\x00\x00\x00\x00\x00"
|
||||
response = b"\x05\x02" + b"\x01\x00" + b"\x05\x00\x00" + bind_addr
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
await socks5_connect(reader, writer, "example.com", 80, "user", "pass")
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_auth_failure(self):
|
||||
"""SOCKS5 auth rejected by server."""
|
||||
response = b"\x05\x02" + b"\x01\x01"
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
with pytest.raises(ProtoError, match="authentication failed"):
|
||||
await socks5_connect(reader, writer, "example.com", 80, "user", "bad")
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_no_acceptable_methods(self):
|
||||
"""Server rejects all auth methods (0xFF)."""
|
||||
response = b"\x05\xff"
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
with pytest.raises(ProtoError, match="no acceptable"):
|
||||
await socks5_connect(reader, writer, "example.com", 80)
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_connect_refused(self):
|
||||
"""SOCKS5 connect reply with connection refused."""
|
||||
bind_addr = b"\x01\x00\x00\x00\x00\x00\x00"
|
||||
response = b"\x05\x00" + b"\x05\x05\x00" + bind_addr
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
with pytest.raises(ProtoError, match="connect failed"):
|
||||
await socks5_connect(reader, writer, "example.com", 80)
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_wrong_version(self):
|
||||
"""Server responds with wrong SOCKS version."""
|
||||
response = b"\x04\x00"
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
with pytest.raises(ProtoError, match="unexpected version"):
|
||||
await socks5_connect(reader, writer, "example.com", 80)
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_server_requires_auth_no_creds(self):
|
||||
"""Server demands auth but no credentials provided."""
|
||||
response = b"\x05\x02"
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
with pytest.raises(ProtoError, match="requires auth"):
|
||||
await socks5_connect(reader, writer, "example.com", 80)
|
||||
|
||||
_run(run())
|
||||
|
||||
|
||||
# -- socks4_connect ----------------------------------------------------------
|
||||
|
||||
|
||||
class TestSocks4Connect:
|
||||
"""Test SOCKS4/4a request building."""
|
||||
|
||||
def test_ip_success(self):
|
||||
"""SOCKS4 connect with IP address."""
|
||||
response = b"\x00\x5a" + b"\x00" * 6
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
await socks4_connect(reader, writer, "1.2.3.4", 80)
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_domain_success(self):
|
||||
"""SOCKS4a connect with domain name."""
|
||||
response = b"\x00\x5a" + b"\x00" * 6
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
await socks4_connect(reader, writer, "example.com", 80)
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_rejected(self):
|
||||
"""SOCKS4 request rejected."""
|
||||
response = b"\x00\x5b" + b"\x00" * 6
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
with pytest.raises(ProtoError, match="rejected"):
|
||||
await socks4_connect(reader, writer, "1.2.3.4", 80)
|
||||
|
||||
_run(run())
|
||||
|
||||
|
||||
# -- http_connect ------------------------------------------------------------
|
||||
|
||||
|
||||
class TestHttpConnect:
|
||||
"""Test HTTP CONNECT request building."""
|
||||
|
||||
def test_success_200(self):
|
||||
"""HTTP CONNECT with 200 response."""
|
||||
response = b"HTTP/1.1 200 Connection Established\r\n\r\n"
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
await http_connect(reader, writer, "example.com", 443)
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_success_with_headers(self):
|
||||
"""HTTP CONNECT with extra headers in response."""
|
||||
response = b"HTTP/1.1 200 OK\r\nX-Proxy: test\r\n\r\n"
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
await http_connect(reader, writer, "example.com", 443)
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_auth_success(self):
|
||||
"""HTTP CONNECT with proxy authentication."""
|
||||
response = b"HTTP/1.1 200 OK\r\n\r\n"
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
await http_connect(reader, writer, "example.com", 443, "user", "pass")
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_forbidden(self):
|
||||
"""HTTP CONNECT with 403 response."""
|
||||
response = b"HTTP/1.1 403 Forbidden\r\n\r\n"
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
with pytest.raises(ProtoError, match="connect failed"):
|
||||
await http_connect(reader, writer, "example.com", 443)
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_proxy_auth_required(self):
|
||||
"""HTTP CONNECT with 407 response."""
|
||||
response = b"HTTP/1.1 407 Proxy Authentication Required\r\n\r\n"
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(response)
|
||||
with pytest.raises(ProtoError, match="connect failed"):
|
||||
await http_connect(reader, writer, "example.com", 443)
|
||||
|
||||
_run(run())
|
||||
|
||||
def test_empty_response(self):
|
||||
"""HTTP CONNECT with empty response."""
|
||||
|
||||
async def run():
|
||||
reader, writer = _make_streams(b"")
|
||||
with pytest.raises(ProtoError, match="empty response"):
|
||||
await http_connect(reader, writer, "example.com", 443)
|
||||
|
||||
_run(run())
|
||||
|
||||
|
||||
# -- Socks5Reply enum -------------------------------------------------------
|
||||
|
||||
|
||||
class TestSocks5Reply:
|
||||
"""Test SOCKS5 reply code values."""
|
||||
|
||||
def test_succeeded(self):
|
||||
assert Socks5Reply.SUCCEEDED == 0x00
|
||||
|
||||
def test_general_failure(self):
|
||||
assert Socks5Reply.GENERAL_FAILURE == 0x01
|
||||
|
||||
def test_connection_refused(self):
|
||||
assert Socks5Reply.CONNECTION_REFUSED == 0x05
|
||||
|
||||
def test_command_not_supported(self):
|
||||
assert Socks5Reply.COMMAND_NOT_SUPPORTED == 0x07
|
||||
|
||||
def test_address_type_not_supported(self):
|
||||
assert Socks5Reply.ADDRESS_TYPE_NOT_SUPPORTED == 0x08
|
||||
|
||||
|
||||
# -- ProtoError --------------------------------------------------------------
|
||||
|
||||
|
||||
class TestProtoError:
|
||||
"""Test ProtoError exception."""
|
||||
|
||||
def test_default_reply(self):
|
||||
err = ProtoError("test error")
|
||||
assert str(err) == "test error"
|
||||
assert err.reply == Socks5Reply.GENERAL_FAILURE
|
||||
|
||||
def test_custom_reply(self):
|
||||
err = ProtoError("refused", Socks5Reply.CONNECTION_REFUSED)
|
||||
assert err.reply == Socks5Reply.CONNECTION_REFUSED
|
||||
|
||||
Reference in New Issue
Block a user