Username
821ade95ef
worker: add local proxy test cache
...
CI / syntax-check (push) Failing after 1s
CI / memory-leak-check (push) Successful in 16s
Skip redundant proxy tests across URL batches using a memory-only
TTL cache (default 300s, configurable via worker.cache_ttl).
2026-02-18 01:37:09 +01:00
Username
af6e27bd77
config: allow ppf.threads = 0 to disable URL cycling
CI / syntax-check (push) Failing after 1s
CI / memory-leak-check (push) Successful in 17s
2026-02-18 00:33:16 +01:00
Username
82c909d7c0
rename --worker-v2 to --worker
...
No V1 means no need for the suffix. Update flag, function name,
compose command, log messages, and docs.
2026-02-17 22:30:09 +01:00
Username
224d3642f9
config: remove V1 worker options
...
Drop --worker flag, batch_size, and claim_timeout. V2 is the
only worker protocol; --worker-v2 and --register remain.
2026-02-17 22:13:31 +01:00
Username
31bdb76a97
config: add fingerprint option for protocol probing
2026-02-17 21:06:10 +01:00
Username
716d60898b
config: allow checktype = none to disable secondary check
...
Accepts none/false/off/disabled as checktype value, normalized to
'none' internally. When set, ssl_first is forced on and no Phase 2
check runs -- only successful TLS handshakes count as working.
2026-02-17 18:37:44 +01:00
Username
4a5210f9f7
config: add worker V2 config items and --worker-v2 flag
2026-02-17 14:23:13 +01:00
Username
8272cf06e0
config: add verification and schedule settings
...
- [verification] section: enabled, threads, batch_size, interval, max_queue
- working_checktime: retest interval for working proxies (default: 300s)
- fail_retry_interval: retry interval for failing proxies (default: 60s)
- fail_retry_backoff: linear backoff for failures (default: True)
- ssl_only: skip secondary check on SSL failure (default: False)
2026-01-08 09:05:20 +01:00
Username
9b44043237
add ssl_first: try SSL handshake before secondary check
...
CI / syntax-check (push) Successful in 3s
CI / memory-leak-check (push) Successful in 11s
When ssl_first=1 (default), proxy validation first attempts an SSL
handshake. If it fails, falls back to the configured secondary check
(head, judges, or irc). This separates SSL capability detection from
basic connectivity testing.
New config options:
- ssl_first: enable SSL-first pattern (default: 1)
- checktype: secondary check type (head, judges, irc)
2025-12-28 14:56:46 +01:00
Username
1c8e3062b7
add configurable thread scaling and queue counter reset
...
CI / syntax-check (push) Successful in 3s
CI / memory-leak-check (push) Successful in 12s
- Add scale_cooldown and scale_threshold config options
- ThreadScaler now reads scaling params from config
- Reset priority queue counter when queue empties to prevent unbounded growth
2025-12-28 14:37:32 +01:00
Username
9f81e1e4af
watchd: remove use_ssl config, use HTTP for non-ssl checks
CI / syntax-check (push) Successful in 2s
CI / memory-leak-check (push) Successful in 11s
2025-12-26 21:25:52 +01:00
Username
6ea201805f
watchd: configurable adaptive timeout per failure
2025-12-26 20:57:21 +01:00
Username
adbe20dae3
cleanup: remove unused imports and dead code
2025-12-26 19:34:23 +01:00
Username
560230988d
support multiple checktypes with random selection
2025-12-25 20:23:05 +01:00
Username
269fed55ff
refactor core modules, integrate network stats
2025-12-25 11:13:20 +01:00
Username
71fb3800ee
config: add gevent dependency and min_threads option
...
- requirements.txt: add gevent for cooperative concurrency
- config.py: add min_threads setting for thread scaling
2025-12-23 17:22:52 +01:00
Username
d48cc3f9eb
config: fix defaults for database and checktype
2025-12-21 23:37:09 +01:00
Username
e24f68500c
style: normalize indentation and improve code style
...
- convert tabs to 4-space indentation
- add docstrings to modules and classes
- remove unused import (copy)
- use explicit object inheritance
- use 'while True' over 'while 1'
- use 'while args' over 'while len(args)'
- use '{}' over 'dict()'
- consistent string formatting
- Python 2/3 compatible Queue import
2025-12-20 23:18:45 +01:00
Username
eeb71a1d55
config: add LibreTranslate settings
...
- libretranslate_url: API endpoint (default: https://lt.mymx.me/translate )
- libretranslate_enabled: toggle for dynamic translations (default: True)
2025-12-20 22:27:45 +01:00
Username
0fd8424d33
config: add --nobs flag to disable BeautifulSoup
2025-12-20 17:33:39 +01:00
Your Name
7cc02fc0a4
config.py: add config.watchd.checktype, to select irc or http check
2021-07-03 18:47:34 +02:00
Your Name
6b6cd94cec
spaces to tabs
2021-06-27 12:31:15 +02:00
Your Name
e236c7cdb1
config.py: add more useless thing for proxyflood
2021-06-27 12:12:04 +02:00
Your Name
9efeff6ec1
changes
2021-06-02 14:54:46 +02:00
Your Name
d3d83e1d90
changes
2021-05-12 08:06:03 +02:00
Your Name
cae6f75643
changs
2021-05-02 00:22:12 +02:00
Your Name
f7e5a5e3b6
add more flood stuff
2021-04-27 13:45:24 +02:00
Your Name
3ab7d9c04d
rename flood.max_time -> flood.duration
2021-04-26 03:32:05 +02:00
Your Name
89e73a818a
add more flood stuff
2021-04-26 03:29:43 +02:00
Your Name
3f7ec14931
add options to floodbot
2021-04-26 01:53:38 +02:00
Your Name
b29c734002
fix: url → self.url, make thread option configurable
2021-02-06 14:33:44 +01:00
Your Name
78b29a1187
some changes
2021-01-24 03:52:56 +01:00
Mickaël Serneels
eeedf9d0a1
extract url only from same domains ? (default: False)
...
setting this option will make ppf not follow external links when extracting uris
2019-05-14 21:24:29 +02:00
Mickaël Serneels
482cf79676
scraper: make query configurable (Proxies, Websites, Search)
...
--scraper.query = 'pws'
2019-05-01 17:43:28 +02:00
Mickaël Serneels
c194d5cfc7
scraper: add debug option
2019-05-01 17:43:28 +02:00
Mickaël Serneels
75318209ab
oldies_multi: change default value from 100 to 10
2019-05-01 17:43:28 +02:00
Mickaël Serneels
bad4d25bcf
make watchd.tor_safeguard a configurable option (default: True)
2019-05-01 17:43:28 +02:00
Mickaël Serneels
003a9074d2
make server file configurable
2019-05-01 17:43:28 +02:00
mickael
310b01140a
irc: implement use_ssl = 2
...
0: disabled, 1: enabled, 2: maybe
default is 0
2019-05-01 17:43:28 +02:00
mickael
1ad5ca53e5
take care of old proxies
...
test old proxies during free time
2019-03-05 22:29:16 +00:00
rofl0r
ddee92d20f
watchd: introduce configurable 'outage_threshold'
2019-01-18 15:34:49 +00:00
rofl0r
24d2c08c9f
ppf: make it possible to import a file containing proxies directly
...
using --file filename.html
2019-01-11 05:45:13 +00:00
rofl0r
8b10df9c1b
ppf.py: start using stale_count
2019-01-11 05:08:32 +00:00
rofl0r
d2cb7441a8
ppf: add optional debug output
2019-01-11 05:03:40 +00:00
mickael
4c6a83373f
split databases
2019-01-11 00:25:01 +00:00
rofl0r
5e774b4e2a
config.py: put section name in var
...
avoids errors due to typos
2019-01-11 00:25:01 +00:00
rofl0r
ef9158015f
proxywatchd: make checktime constants configurable
...
this requires only saving the last checked time in `tested`.
you can run the following sql statement to update the existing values
in the database:
sqlite3 proxylist.sqlite \
"update proxylist set tested=tested-(1800+(failed*3600)) where failed < 6"
2019-01-11 00:25:01 +00:00
rofl0r
38d89f5bd9
ppf: add option for number of http retries
2019-01-08 03:30:31 +00:00
rofl0r
f16f754b0e
implement combo config parser
...
allows all options to be overridden by command line.
e.g.
[watchd]
threads=10
debug=false
--watch.threads=50 --debug=true
2019-01-08 02:17:10 +00:00
mickael
1b3ce72efc
add and use combining class
2019-01-07 23:19:14 +00:00
