Username
3ac7305954
proxywatchd: persist MITM certificate stats across restarts
...
Add save_state/load_state to MITMCertStats for JSON persistence.
Stats saved periodically (5min) and at shutdown, loaded at startup.
2025-12-25 19:47:51 +01:00
Username
272eba0f05
scraper: reuse connections, cycle circuit on block
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 15s
2025-12-25 19:26:23 +01:00
Username
68e8b88afa
tor: use random credentials for circuit isolation
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-25 19:18:25 +01:00
Username
9ba965c87f
proxywatchd: ensure socket cleanup before SSL fallback
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-25 19:13:47 +01:00
Username
b41d077905
proxywatchd: make peak_rate per-session, not persistent
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-25 18:54:05 +01:00
Username
92b4036c37
proxywatchd: delay peak measurement until after startup
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-25 18:48:27 +01:00
Username
a8d06666b7
proxywatchd: fallback to HTTP when SSL fails
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-25 18:39:51 +01:00
Username
c459736561
proxywatchd: add sanity checks to rate/peak calculations
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-25 18:11:22 +01:00
Username
9ebfa01185
docs: add project instructions
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-25 11:14:41 +01:00
Username
d09f6d5e08
docs: update roadmap and todo
2025-12-25 11:14:27 +01:00
Username
5e184c1130
update config sample, requirements, searx instances
2025-12-25 11:13:34 +01:00
Username
269fed55ff
refactor core modules, integrate network stats
2025-12-25 11:13:20 +01:00
Username
2201515b10
proxywatchd: track failures by protocol and SSL category
2025-12-25 02:51:47 +01:00
Username
9429d24fd5
httpd: extract static files to separate directory
2025-12-25 02:51:30 +01:00
Username
630ed96aa2
engines: add Bing and Yahoo search engines
2025-12-25 02:51:11 +01:00
Username
26f00c20de
add network statistics tracking module
2025-12-25 02:49:34 +01:00
Username
372d7643cc
httpd: add country pie chart to dashboard
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-24 01:37:57 +01:00
Username
9738cc9664
httpd: consolidate map page with shared css
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-24 01:34:33 +01:00
Username
38fb16a439
proxywatchd: fix mitm reset logic, track ssl_mitm category
2025-12-24 01:26:37 +01:00
Username
1a40f63bac
docs: fix project name to Python Proxy Finder
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-24 01:23:04 +01:00
Username
4f17e7c546
proxywatchd: treat MITM as working, mark fatal errors as dead
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-24 01:17:02 +01:00
Username
97a7dc3316
fetch: use raw strings for regex patterns
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 14s
2025-12-24 01:06:49 +01:00
Username
c6dca3e51b
ci: switch to python3 for all checks
CI / syntax-check (push) Successful in 6s
CI / memory-leak-check (push) Successful in 18s
2025-12-24 01:00:28 +01:00
Username
ec861d94c4
ci: fix yaml syntax in import check step
CI / syntax-check (push) Failing after 5s
CI / memory-leak-check (push) Failing after 12s
2025-12-24 00:58:47 +01:00
Username
6e7c78f0dd
ci: fix runner label to dotfiles
2025-12-24 00:55:51 +01:00
Username
048c7e78ef
ci: add workflow_dispatch trigger
2025-12-24 00:26:10 +01:00
Username
0b26cec827
docs: update todo with profiling optimizations
2025-12-24 00:20:40 +01:00
Username
d8852753cc
docs: add dashboard documentation
2025-12-24 00:20:29 +01:00
Username
9360c35add
ppf: add format_duration helper and stale log improvements
...
- Add format_duration() for compact time display
- Improve stale proxy logging with duration info
2025-12-24 00:20:13 +01:00
Username
5e788c06d1
fetch: precompile proxy extraction regex
...
Move regex pattern compilation to module load time
for better performance in repeated calls.
2025-12-24 00:20:06 +01:00
Username
33f9a211ce
scraper: fix return values and add stats logging
...
- Return counts from proxyfind for tracking
- Add 15-minute stats interval logging
- Track new_urls and queries count
2025-12-24 00:19:53 +01:00
Username
689ea8153b
dbs: add batch operations and session state persistence
...
- Add batch_update_proxy_latency for bulk updates
- Add save/load_session_state for stats persistence
- Improve insert_urls to return new count
2025-12-24 00:19:46 +01:00
Username
9ed3287e41
ci: add gitea workflow for syntax and memory leak checks
...
- Python 2/3 syntax validation
- Static analysis for memory leak patterns
- Unbounded collection detection
- Circular reference checks
2025-12-24 00:19:13 +01:00
Username
f2e6b8216b
httpd: fix dashboard accuracy and add memory tracking
...
- Add from __future__ import division
- Change "Tests This Session" to "Tests (Cumulative)"
- Fix Tor status to show IDLE when available but 0% success
- Add memory leak detection: RSS growth, peak, GC stats
2025-12-24 00:19:05 +01:00
Username
de750a1312
proxywatchd: fix division, uptime, and scaler
...
- Add from __future__ import division for correct percentages
- Remove start_time restoration so uptime reflects current session
- Remove success_rate threshold for scaling (scale on queue depth only)
2025-12-24 00:17:47 +01:00
Username
738fccdabb
connection_pool: fix python 2 integer division
...
Add from __future__ import division to fix success_rate
and latency calculations returning 0 due to integer division.
2025-12-24 00:17:40 +01:00
Username
f83733dd46
proxywatchd: mark confirmed-dead proxies as permanently dead
...
- Add DEAD_PROXY=-1 constant for permanently dead proxies
- Mark proxy dead when: failed >= max_fail*2, or max_fail with fatal error
- Fatal errors: refused, unreachable, auth (proxy definitely not working)
- Dead proxies excluded from testing (failed >= 0 query)
- Cleanup_stale also removes old dead proxies
- Dashboard shows separate dead vs failing counts
2025-12-23 18:03:01 +01:00
Username
1e2054bec5
dashboard: fix missing tor pool and db stats fields
2025-12-23 17:56:12 +01:00
Username
6076d93377
docs: update roadmap and todo with dashboard v2 features
2025-12-23 17:48:59 +01:00
Username
53f37510f3
dashboard: add system monitoring and enhanced stats
...
- prominent check type badge in header (SSL/judges/http/irc)
- system monitor bar: load, memory, disk, process RSS
- anonymity breakdown: elite/anonymous/transparent counts
- database health: size, recent activity, dead proxy count
- enhanced Tor pool stats: requests, success rate, latency
- SQLite ANALYZE/VACUUM functions for query optimization
- database statistics API functions
2025-12-23 17:47:12 +01:00
Username
20fc1b01fd
add export.py for proxy list export
...
CLI tool for exporting working proxies:
- multiple formats: txt, json, csv, length-prefixed
- filters: protocol, country, anonymity, max latency
- sorting: latency, added time, success count
- configurable output limit
Also update .gitignore to exclude data/ directory
2025-12-23 17:34:51 +01:00
Username
eb1bba0e13
docs: update roadmap and task tracking
...
- README: update feature list
- ROADMAP: add completed features, update priorities
- TODO: mark completed tasks, add new items
- config.ini.sample: update example values
- http2: minor cleanup
2025-12-23 17:24:25 +01:00
Username
1c437d2246
httpd: modern dashboard with gevent WSGI
...
- dark theme with CSS custom properties
- responsive grid layout for stats cards
- sparkline charts for rate/success history
- gevent pywsgi server with BaseHTTPServer fallback
- new API endpoints: /api/stats, /api/judges, /api/protocols
- protocol breakdown, latency percentiles, geo distribution
- tor pool health, judge availability, engine stats
- auto-refresh with configurable interval
2025-12-23 17:23:58 +01:00
Username
c0dbba7b45
proxywatchd: gevent concurrency and comprehensive stats
...
Major features:
- gevent monkey-patching for cooperative concurrency
- Stats class with latency percentiles, histograms, geo tracking
- session state persistence across restarts
- JudgeStats with cooldown for blocked/rate-limited judges
- ThreadScaler for dynamic greenlet pool sizing
- SIGHUP handler for config hot-reload
- SSL/TLS tracking with MITM detection
- anonymity detection (transparent/anonymous/elite)
Bug fixes:
- rwip(): fix string comparison (b[0] == '0' not == 0)
- rwip(): fix strip logic (b[1:] not b[:1])
- use string.ascii_letters for Py2/3 compatibility
2025-12-23 17:23:45 +01:00
Username
e7478de79e
scraper: add engine stats API for dashboard
...
- EngineTracker.get_stats() returns detailed per-engine metrics
- get_scraper_stats() module function for external access
- includes success counts, backoff status, availability
2025-12-23 17:23:28 +01:00
Username
68a34f2638
fetch: detect proxy protocol from source URL path
...
- detect_proto_from_path() infers socks4/socks5/http from URL
- extract_proxies() now returns (address, proto) tuples
- ppf.py updated to handle protocol-tagged proxies
- profiler signal handler for SIGTERM stats dump
2025-12-23 17:23:17 +01:00
Username
e0e330301a
dbs: add session persistence and stats history
...
- session_state table for persisting runtime stats across restarts
- stats_history table for hourly snapshots (24h graphs)
- latency tracking with exponential moving average
- anonymity detection columns (transparent/anonymous/elite)
- curated PROXY_SOURCES list for seeding
- migration functions for existing databases
2025-12-23 17:23:04 +01:00
Username
71fb3800ee
config: add gevent dependency and min_threads option
...
- requirements.txt: add gevent for cooperative concurrency
- config.py: add min_threads setting for thread scaling
2025-12-23 17:22:52 +01:00
Username
267035802a
ppf: reset stale_count when content hash changes
2025-12-22 00:05:06 +01:00
Username
f382a4ab6a
ppf: add content hash for duplicate proxy list detection
2025-12-22 00:03:12 +01:00
