452 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Username	0d7d2dce70	refactor: extract modules from proxywatchd.py ... Extract focused modules to reduce proxywatchd.py complexity: - stats.py: JudgeStats, Stats, regexes, ssl_targets (557 lines) - mitm.py: MITMCertStats, cert extraction functions (239 lines) - dns.py: socks4_resolve with TTL caching (86 lines) - job.py: PriorityJobQueue, calculate_priority (103 lines) proxywatchd.py reduced from 2488 to 1591 lines (-36%).	2025-12-28 15:45:24 +01:00
Username	35f24bb8b0	dashboard: refactor layout and add worker stats	2025-12-28 15:19:50 +01:00
Username	e89db20f5b	scraper: add Bing and Yahoo engines	2025-12-28 15:19:39 +01:00
Username	0fbfee2855	httpd: add worker registration and distributed testing API	2025-12-28 15:19:08 +01:00
Username	3b361916fa	fetch, dbs: minor refactoring	2025-12-28 15:18:42 +01:00
Username	50bc13a615	misc: add SSL protocol error detection and diag logging	2025-12-28 15:18:29 +01:00
Username	f4286ea515	ppf: remove num_targets param (removed in phase 2)	2025-12-28 15:16:52 +01:00
Username	9b44043237	add ssl_first: try SSL handshake before secondary check ... When ssl_first=1 (default), proxy validation first attempts an SSL handshake. If it fails, falls back to the configured secondary check (head, judges, or irc). This separates SSL capability detection from basic connectivity testing. New config options: - ssl_first: enable SSL-first pattern (default: 1) - checktype: secondary check type (head, judges, irc)	2025-12-28 14:56:46 +01:00
Username	9f782c3222	config: default to head checktype only	2025-12-28 14:45:27 +01:00
Username	1c8e3062b7	add configurable thread scaling and queue counter reset ... - Add scale_cooldown and scale_threshold config options - ThreadScaler now reads scaling params from config - Reset priority queue counter when queue empties to prevent unbounded growth	2025-12-28 14:37:32 +01:00
Username	d219cc567f	phase 2: code cleanup and simplification ... - Remove unused result_queue from WorkerThread and worker mode - Remove num_targets abstraction, simplify to single-target mode - Add _db_context() context manager for database connections - Refactor 5 call sites to use context manager (finish, init, cleanup_stale, periodic saves) - Mark _prep_db/_close_db as deprecated - Add __version__ = '2.0.0' to ppf.py - Add thread spawn stagger (0-100ms) in worker mode for Tor-friendly startup	2025-12-28 14:31:37 +01:00
Username	72a2dcdaf4	ppf: add worker mode with distributed testing ... - Add --worker mode for distributed proxy testing - Workers claim batches from manager, test via local Tor, submit results - Add --register to register new workers with manager - Add thread spawn stagger (0-100ms) to avoid overwhelming Tor - Verify Tor connectivity before claiming work - Add heartbeat and batch timeout handling - Track worker profiling state for dashboard display	2025-12-28 14:12:59 +01:00
Username	e46a3f6ef1	proxywatchd: add DNS cache TTL, IP validation, debug logging ... - DNS cache entries now expire after 1 hour (DNS_CACHE_TTL) - Add is_valid_ip() to validate extracted IPs have valid octets - Add debug logging to silent exception blocks (ASN lookup, socket disconnect) - Convert rwip() to static method with cleaner int() implementation - Add _sample_dbg() for sampled diagnostic logging - Add set_config() for worker mode imports - Remove unused tor_targets (tor checktype was removed) - Move tor_proxy_url to misc.py for shared use	2025-12-28 14:12:50 +01:00
Username	58d83ae52f	dashboard: always show SSL badge	2025-12-26 21:37:37 +01:00
Username	d9205614fc	proxywatchd: secondary check only for SSL handshake failures	2025-12-26 21:36:45 +01:00
Username	03dd277c54	dashboard: fix ssl badge after use_ssl config removal	2025-12-26 21:30:23 +01:00
Username	9f81e1e4af	watchd: remove use_ssl config, use HTTP for non-ssl checks	2025-12-26 21:25:52 +01:00
Username	f7a762331a	watchd: add tor checktype, use Tor API for secondary check	2025-12-26 21:20:16 +01:00
Username	d2bd7d4f34	fetch: retry with different Tor circuit on failure	2025-12-26 20:57:28 +01:00
Username	6ea201805f	watchd: configurable adaptive timeout per failure	2025-12-26 20:57:21 +01:00
Username	7232846b0f	ppf: add --reset flag to clear all state	2025-12-26 20:57:15 +01:00
Username	906d1b33ae	fetch: cache is_usable_proxy results	2025-12-26 20:04:01 +01:00
Username	1f41f3df5c	dashboard: pause polling when tab is hidden	2025-12-26 19:58:20 +01:00
Username	ad89eb262e	httpd: add rate limiting and security headers	2025-12-26 19:52:45 +01:00
Username	59fe2c4a14	dashboard: support multiple checktype badges	2025-12-26 19:34:23 +01:00
Username	adbe20dae3	cleanup: remove unused imports and dead code	2025-12-26 19:34:23 +01:00
Username	a20b5525f0	ppf: handle confidence field in proxy tuples	2025-12-26 19:34:22 +01:00
Username	481dc514fb	fetch: add IPv6, auth proxy, and confidence scoring support	2025-12-26 19:13:36 +01:00
Username	50f49a20ff	httpd: restrict /api/memory to localhost only	2025-12-26 19:12:39 +01:00
Username	07262e8b50	proxywatchd: cap threads by queue size, faster scale-down	2025-12-26 18:12:39 +01:00
Username	8ae639cb94	httpd: cache gc.get_objects and get_db_health calls	2025-12-26 18:12:31 +01:00
Username	b955281b51	config: set checktype to judges,head	2025-12-25 20:25:43 +01:00
Username	560230988d	support multiple checktypes with random selection	2025-12-25 20:23:05 +01:00
Username	755abc7f6e	dashboard: add queue ETA countdown to Worker Pool pane	2025-12-25 19:58:49 +01:00
Username	0424527e84	proxywatchd: add 0.3s delay before SSL fallback to HTTP	2025-12-25 19:49:59 +01:00
Username	3ac7305954	proxywatchd: persist MITM certificate stats across restarts ... Add save_state/load_state to MITMCertStats for JSON persistence. Stats saved periodically (5min) and at shutdown, loaded at startup.	2025-12-25 19:47:51 +01:00
Username	272eba0f05	scraper: reuse connections, cycle circuit on block	2025-12-25 19:26:23 +01:00
Username	68e8b88afa	tor: use random credentials for circuit isolation	2025-12-25 19:18:25 +01:00
Username	9ba965c87f	proxywatchd: ensure socket cleanup before SSL fallback	2025-12-25 19:13:47 +01:00
Username	b41d077905	proxywatchd: make peak_rate per-session, not persistent	2025-12-25 18:54:05 +01:00
Username	92b4036c37	proxywatchd: delay peak measurement until after startup	2025-12-25 18:48:27 +01:00
Username	a8d06666b7	proxywatchd: fallback to HTTP when SSL fails	2025-12-25 18:39:51 +01:00
Username	c459736561	proxywatchd: add sanity checks to rate/peak calculations	2025-12-25 18:11:22 +01:00
Username	9ebfa01185	docs: add project instructions	2025-12-25 11:14:41 +01:00
Username	d09f6d5e08	docs: update roadmap and todo	2025-12-25 11:14:27 +01:00
Username	5e184c1130	update config sample, requirements, searx instances	2025-12-25 11:13:34 +01:00
Username	269fed55ff	refactor core modules, integrate network stats	2025-12-25 11:13:20 +01:00
Username	2201515b10	proxywatchd: track failures by protocol and SSL category	2025-12-25 02:51:47 +01:00
Username	9429d24fd5	httpd: extract static files to separate directory	2025-12-25 02:51:30 +01:00
Username	630ed96aa2	engines: add Bing and Yahoo search engines	2025-12-25 02:51:11 +01:00