From a1fc19fb451bff14448c898567ec9f3852280f37 Mon Sep 17 00:00:00 2001 From: user Date: Sun, 15 Feb 2026 22:20:17 +0100 Subject: [PATCH] docs: update for codebase consolidation and startup fixes - Remove source.py from architecture (deleted) - Add metrics.py to module list - Update warm start: trusts cached state, instant startup - Update signal handling: registered before startup - Add refactoring tasks to TASKS.md - Remove stale troubleshooting entry Co-Authored-By: Claude Opus 4.6 --- PROJECT.md | 8 ++++---- ROADMAP.md | 2 +- TASKS.md | 14 +++++++++++++- docs/CHEATSHEET.md | 3 +-- docs/USAGE.md | 9 ++++----- 5 files changed, 23 insertions(+), 13 deletions(-) diff --git a/PROJECT.md b/PROJECT.md index 15952a7..0ef26f8 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -21,12 +21,12 @@ Client -------> s5p -------> Hop 1 -------> Hop 2 -------> Target - **server.py** -- asyncio SOCKS5 server, bidirectional relay, signal handling - **proto.py** -- protocol handshakes (SOCKS5, SOCKS4/4a, HTTP CONNECT), chain builder -- **config.py** -- YAML config loading, proxy URL parsing, pool config +- **config.py** -- YAML config loading, proxy URL parsing, API response parsing, pool config - **pool.py** -- managed proxy pool (multi-source, health-tested, persistent) -- **source.py** -- legacy proxy source (single HTTP API, kept for backward compat) - **http.py** -- minimal async HTTP/1.1 client (GET/POST JSON, no external deps) - **connpool.py** -- pre-warmed TCP connection pool to first chain hop - **cli.py** -- argparse CLI, logging setup, cProfile support +- **metrics.py** -- connection counters and human-readable summary (lock-free, asyncio-only) ## Deployment @@ -53,14 +53,14 @@ All other functionality uses Python stdlib (`asyncio`, `socket`, `struct`). - **asyncio** -- single-threaded event loop, efficient for I/O-bound proxying - **Domain passthrough** -- never resolve DNS locally to prevent leaks - **Tor as a hop** -- no special Tor handling; it's just `socks5://127.0.0.1:9050` -- **Graceful shutdown** -- SIGTERM/SIGINT handled in the event loop for clean container stops +- **Graceful shutdown** -- SIGTERM/SIGINT registered before startup for clean container stops - **Config split** -- tracked example template, gitignored live config with real addresses - **Proxy pool** -- multi-source (API + file), health-tested, persistent, auto-cleaned - **Weighted selection** -- recently-tested proxies preferred via recency decay weight - **Failure backoff** -- connection failures penalize proxy weight for 60s, avoids retry waste - **Stale expiry** -- proxies dropped from sources evicted after 3 refresh cycles if not alive - **Chain pre-flight** -- static chain tested before pool health tests; skip on failure -- **Warm start** -- quick-test alive subset on restart, defer full test to background +- **Warm start** -- trust cached alive state on restart, defer all health tests to background - **SIGHUP reload** -- re-read config, update pool settings, re-fetch sources - **Dead reporting** -- POST evicted proxies to upstream API for list quality feedback - **Connection semaphore** -- cap concurrent connections to prevent fd exhaustion diff --git a/ROADMAP.md b/ROADMAP.md index 8208f8c..a66082e 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -15,7 +15,7 @@ - [x] Per-proxy backoff (connection failure cooldown) - [x] Stale proxy expiry (last_seen TTL) - [x] Pool stats in periodic metrics log -- [x] Fast warm start (deferred full health test) +- [x] Instant warm start (trust cached state, defer all health tests) - [x] Static chain health check (pre-flight before pool tests) - [x] SIGHUP hot config reload - [x] Dead proxy reporting to source API diff --git a/TASKS.md b/TASKS.md index 2e4279f..1154a94 100644 --- a/TASKS.md +++ b/TASKS.md @@ -23,7 +23,7 @@ - [x] Per-proxy backoff (60s cooldown after connection failure) - [x] Stale proxy expiry (evict dead proxies not seen for 3 refresh cycles) - [x] Pool stats in periodic metrics log (`pool=alive/total`) -- [x] Fast warm start (quick-test alive subset, defer full test) +- [x] Fast warm start (trust cached state, defer all health tests) - [x] Static chain health check (skip pool tests if chain unreachable) - [x] SIGHUP hot config reload (timeout, retries, log_level, pool config) - [x] Dead proxy reporting (`report_url` POST evicted proxies to API) @@ -31,6 +31,18 @@ - [x] Async HTTP client (replace blocking urllib, parallel source fetch) - [x] First-hop TCP connection pool (`pool_size`, `pool_max_idle`) +- [x] Codebase consolidation (refactor/codebase-consolidation) + - [x] Extract shared proxy parsing and constants to config.py + - [x] Consolidate health-check HTTP logic in pool + - [x] Remove threading from metrics (pure asyncio, no lock needed) + - [x] Replace `ensure_future` with `create_task` + - [x] Rename ambiguous variables in config loader + - [x] Remove legacy ProxySource layer (source.py deleted) + - [x] Add tests for extracted `parse_api_proxies` +- [x] Instant warm start (trust cached state, defer all health tests) +- [x] Register signal handlers before startup (fix SIGKILL on stop) +- [x] Use k8s-file logging driver with rotation + ## Next - [ ] Integration tests with mock proxy server - [ ] SOCKS5 server-side authentication diff --git a/docs/CHEATSHEET.md b/docs/CHEATSHEET.md index 0ab354f..a945474 100644 --- a/docs/CHEATSHEET.md +++ b/docs/CHEATSHEET.md @@ -123,7 +123,6 @@ metrics: conn=142 ok=98 fail=44 retries=88 active=3 in=1.2M out=4.5M up=0h05m12s | DNS leak | Use `socks5h://` (not `socks5://`) in client | | Auth failed | Verify credentials in proxy URL | | Port in use | `fuser -k 1080/tcp` to free the port | -| Container slow stop | Rebuild image after SIGTERM fix | | Proxy keeps failing | Backoff penalizes for 60s; check `pool=` in metrics | | "static chain unreachable" | Tor/upstream hop is down; pool tests skipped | -| Slow startup | Normal on cold start; warm restarts use state file | +| Slow startup | Normal on cold start; warm restarts serve instantly from state | diff --git a/docs/USAGE.md b/docs/USAGE.md index e0d5566..f5f548e 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -179,11 +179,10 @@ are loaded for fast warm starts. ### Warm start -When restarting with an existing state file, only the previously-alive proxies -are tested before the server starts accepting connections. A full health test -of all proxies runs in the background. This reduces startup blocking from -minutes to seconds on warm restarts. Cold starts (no state file) test all -proxies before serving. +When restarting with an existing state file, the server trusts the cached +alive state and begins accepting connections immediately. A full health test +of all proxies runs in the background. Startup takes seconds regardless of +pool size. Cold starts (no state file) test all proxies before serving. ### Dead proxy reporting