Commit Graph

495 Commits

Author SHA1 Message Date
Username
01b91836c4 tools: fix ansible output filter in ppf-db
Some checks failed
CI / syntax-check (push) Failing after 1s
CI / memory-leak-check (push) Successful in 16s
2026-02-18 01:02:45 +01:00
Username
04fb362181 tools: add ppf-status cluster overview 2026-02-18 01:02:42 +01:00
Username
304830e382 watchd: delete proxies immediately on max_fail instead of marking stale
Some checks failed
CI / syntax-check (push) Failing after 1s
CI / memory-leak-check (push) Successful in 17s
2026-02-18 00:50:00 +01:00
Username
752ef359b5 docs: update odin role to SSL-only verification
Some checks failed
CI / syntax-check (push) Failing after 1s
CI / memory-leak-check (push) Successful in 17s
2026-02-18 00:46:51 +01:00
Username
af6e27bd77 config: allow ppf.threads = 0 to disable URL cycling
Some checks failed
CI / syntax-check (push) Failing after 1s
CI / memory-leak-check (push) Successful in 17s
2026-02-18 00:33:16 +01:00
Username
c091216afc docs: add ppf-db to README operations toolkit
Some checks failed
CI / syntax-check (push) Failing after 1s
CI / memory-leak-check (push) Successful in 16s
2026-02-18 00:29:28 +01:00
Username
4cefdf976c docs: update CLAUDE.md for ppf-db and corrected odin role 2026-02-18 00:29:25 +01:00
Username
98c2e74412 ppf: skip URL cycling when ppf.threads = 0 2026-02-18 00:28:37 +01:00
Username
24d6f345f6 tools: add ppf-db for database operations 2026-02-18 00:28:27 +01:00
Username
1ca096c78a ppf-service: use down+up for restart to pick up code changes 2026-02-18 00:22:55 +01:00
Username
15a7f0bb6a ppf-common: fix compose_cmd to run as podman user 2026-02-18 00:22:52 +01:00
Username
b6045bd05c tools: use down+up in deploy handler to pick up code changes 2026-02-18 00:22:48 +01:00
Username
d7b004f0ac httpd: include protocol in /proxies plain text format 2026-02-18 00:18:58 +01:00
Username
00952b7947 fix: call evaluate() in worker mode before checking results
failcount was initialized to 0 and never updated because
evaluate() was skipped, causing all proxies to pass.
2026-02-18 00:16:35 +01:00
Username
6800995361 docs: reflect podman-compose on all nodes
Remove stale systemd unit and standalone podman run references.
All nodes now managed exclusively via compose.
2026-02-17 23:44:35 +01:00
Username
7a271896a8 ppf-common: fix ad-hoc ansible for toolkit inventory
Add --become to ansible_cmd (needed when connecting as
ansible user). Add cd /tmp to podman_cmd so sudo -u podman
doesn't fail on inaccessible /home/ansible cwd.
2026-02-17 23:38:13 +01:00
Username
8779979780 tools: use compose up -d for ppf-service restart 2026-02-17 23:23:10 +01:00
Username
195d25c653 tools: use compose up -d instead of restart in handler
compose restart reuses the existing container config; up -d
recreates from compose.yml, picking up changes like renamed
CLI flags.
2026-02-17 23:22:33 +01:00
Username
9b8be9d302 tools: use toolkit inventory for all ansible commands
Route ansible_cmd through ppf inventory instead of /opt/ansible
default. Eliminates dynamic inventory warnings and connects
via WireGuard IPs.
2026-02-17 23:22:29 +01:00
Username
9eff4496d6 docs: update README and ROADMAP for playbook deployment 2026-02-17 23:19:59 +01:00
Username
b1de91a969 docs: update CLAUDE.md for playbook-based deployment
Document WireGuard connectivity, playbook architecture, --check
flag, parallel execution, and updated ad-hoc ansible commands
using toolkit inventory.
2026-02-17 23:19:54 +01:00
Username
df2078c7f7 tools: fix symlink resolution in ppf-logs and ppf-service 2026-02-17 23:18:50 +01:00
Username
782deab95d tools: rewrite ppf-deploy as playbook wrapper
Replace sequential ansible ad-hoc calls with ansible-playbook.
Add ansible_playbook_cmd to shared library. Supports --check
for dry runs.
2026-02-17 23:18:46 +01:00
Username
8208670fc1 tools: add ansible deploy playbook
Parallel execution across hosts, handler-based restart on change,
role-aware paths via group_vars. Connects over WireGuard with
dedicated inventory and SSH key.
2026-02-17 23:18:41 +01:00
Username
d902ecafff docs: add tools to ROADMAP.md file reference 2026-02-17 22:53:01 +01:00
Username
fdb761f9f1 docs: add operations toolkit to README.md 2026-02-17 22:52:58 +01:00
Username
12f6b1d8eb docs: update CLAUDE.md for operations toolkit
Replace verbose ansible deployment commands with ppf-deploy,
ppf-logs, and ppf-service references. Keep raw ansible only
for ad-hoc config operations not covered by tools.
2026-02-17 22:52:54 +01:00
Username
1f14173595 tools: add ppf-service
Status, start, stop, restart for PPF containers. Status includes
compose ps, master health check, and worker API query.
2026-02-17 22:50:42 +01:00
Username
2128814a41 tools: add ppf-logs
View container logs with -f follow and -n line count. Resolves
dynamic UID and container name per node role.
2026-02-17 22:50:38 +01:00
Username
7f59cae05c tools: add ppf-deploy
Validates syntax, rsyncs code, copies compose files, fixes ownership,
restarts containers. Supports --no-restart and per-host targeting.
2026-02-17 22:50:34 +01:00
Username
9b7ca20728 tools: add shared library ppf-common.sh
Host topology, ansible/podman/compose wrappers, color helpers,
syntax validation, and target resolution for the PPF ops toolkit.
2026-02-17 22:50:30 +01:00
Username
82c909d7c0 rename --worker-v2 to --worker
No V1 means no need for the suffix. Update flag, function name,
compose command, log messages, and docs.
2026-02-17 22:30:09 +01:00
Username
cb52a978e9 todo: mark V1 worker deprecation complete 2026-02-17 22:13:49 +01:00
Username
224d3642f9 config: remove V1 worker options
Drop --worker flag, batch_size, and claim_timeout. V2 is the
only worker protocol; --worker-v2 and --register remain.
2026-02-17 22:13:31 +01:00
Username
d184dc2926 httpd: remove V1 work distribution and result submission
Drop _work_claims tracking, claim_work(), submit_results(),
get_due_proxy_count(), calculate_fair_batch_size(), and the
/api/work + /api/results endpoint handlers.
2026-02-17 22:12:57 +01:00
Username
2782e6d754 ppf: remove V1 worker functions and main loop
Drop worker_get_work(), worker_submit_results(), and the entire
worker_main() V1 loop. Rewire --register to use worker_v2_main().
2026-02-17 22:10:38 +01:00
Username
0c3c7278f5 rocksock: cache SSL contexts to avoid repeated CA store loads
Some checks failed
CI / syntax-check (push) Failing after 1s
CI / memory-leak-check (push) Successful in 17s
set_default_verify_paths was called per connection (2k+/cycle),
spending ~24s reloading the CA store from disk. Cache two shared
contexts (verify/noverify) at module level instead.
2026-02-17 21:28:36 +01:00
Username
ff21c75a7a add .venv to .gitignore
Some checks failed
CI / syntax-check (push) Failing after 1s
CI / memory-leak-check (push) Successful in 17s
2026-02-17 21:07:16 +01:00
Username
e0e56935f2 todo: add V1 worker deprecation task 2026-02-17 21:06:35 +01:00
Username
9ecf7d89bd servers: refresh list from mirc.com (128 servers) 2026-02-17 21:06:31 +01:00
Username
ba9553f4aa httpd: add freshness filter, mitm param, and provenance to upsert
Export endpoints now require last_seen within 60 minutes. Add optional
mitm=0|1 query parameter to filter MITM proxies. Fix upsert to track
success_count, consecutive_success, last_check, and last_target.
2026-02-17 21:06:27 +01:00
Username
dfcd8f0c00 add test provenance columns and worker report fields
Add last_check/last_target columns to proxylist schema with migration.
Include checktype and target in V2 worker report payload.
2026-02-17 21:06:21 +01:00
Username
4c5f4fa01d watchd: add protocol fingerprint probes and fix nullable counters
Add lightweight SOCKS5/SOCKS4/HTTP handshake probes to identify proxy
protocol before full testing. Guard consecutive_success, success_count,
and total_duration against NoneType from worker-reported upserts.
Track last_check and last_target for test provenance.
2026-02-17 21:06:16 +01:00
Username
31bdb76a97 config: add fingerprint option for protocol probing 2026-02-17 21:06:10 +01:00
Username
672c1bc1f8 docs: update deployment to use podman-compose
Some checks failed
CI / syntax-check (push) Failing after 1s
CI / memory-leak-check (push) Successful in 17s
2026-02-17 18:52:58 +01:00
Username
fab1e1d110 compose: rewrite master and worker compose files
Some checks failed
CI / syntax-check (pull_request) Failing after 0s
CI / syntax-check (push) Failing after 0s
CI / memory-leak-check (pull_request) Failing after 16s
CI / memory-leak-check (push) Successful in 16s
Drop deprecated version key, add SELinux volume labels, SIGTERM
handling with 30s grace period, configurable master URL via
PPF_MASTER_URL env var, and usage documentation in headers.
2026-02-17 18:37:49 +01:00
Username
716d60898b config: allow checktype = none to disable secondary check
Accepts none/false/off/disabled as checktype value, normalized to
'none' internally. When set, ssl_first is forced on and no Phase 2
check runs -- only successful TLS handshakes count as working.
2026-02-17 18:37:44 +01:00
Username
2e3ce149f9 watchd: tighten secondary check validation
- judge blocks record as neutral (judge_block category), not success;
  evaluate() filters them out so they affect neither pass nor fail count
- require HTTP/1.x response line for non-IRC checks; non-HTTP garbage
  (captive portals, proxy error pages) fails immediately
- add is_public_ip() rejecting RFC 1918, loopback, link-local, and
  multicast ranges from judge exit IP extraction
- remove 5 weak HEAD regex targets whose fingerprint headers appear on
  error pages and captive portals (p3p, X-XSS-Protection,
  x-frame-options, referrer-policy, X-UA-Compatible)
2026-02-17 18:37:38 +01:00
Username
1236ddbd2d add compose files for container management
Replace raw podman run with declarative compose.yml per host type.
Master (odin) gets compose.master.yml, workers get compose.worker.yml.
2026-02-17 18:17:12 +01:00
Username
0311abb46a fetch: encode unicode URLs to bytes before HTTP/SOCKS ops
When URLs arrive as unicode (e.g. from JSON API responses), the unicode
type propagates through _parse_url into the SOCKS5 packet construction
in rocksock. Port bytes > 127 formatted via %c in a unicode string
produce non-ASCII characters that fail on socket sendall() implicit
ASCII encode.

Encode URLs to UTF-8 bytes at fetch entry points to keep the entire
request pipeline in str (bytes) domain.
2026-02-17 16:43:26 +01:00