Commit Graph

88 Commits

Author SHA1 Message Date
Username
e24f68500c style: normalize indentation and improve code style
- convert tabs to 4-space indentation
- add docstrings to modules and classes
- remove unused import (copy)
- use explicit object inheritance
- use 'while True' over 'while 1'
- use 'while args' over 'while len(args)'
- use '{}' over 'dict()'
- consistent string formatting
- Python 2/3 compatible Queue import
2025-12-20 23:18:45 +01:00
Username
a694e441a4 proxywatchd: add priority queue for job scheduling 2025-12-20 23:11:49 +01:00
Username
af5e1ce4b0 proxywatchd: integrate tor connection pool 2025-12-20 23:02:26 +01:00
Username
2f2ff9a385 proxywatchd: add stats tracking and httpd integration
- Stats class with failure category tracking
- Configurable stats_interval for periodic reports
- Optional httpd server startup when enabled
- cleanup_stale() for removing dead proxies
2025-12-20 22:28:23 +01:00
Username
86cabd1562 standardize code style: shebangs, class definitions, comments 2025-12-20 18:05:41 +01:00
Username
68d6a8e15f proxywatchd: implement multi-target validation with work-stealing queue 2025-12-20 16:46:09 +01:00
Your Name
d7db366857 split to ip/port, "cleanse" ips and ports, bugfixes 2021-08-22 20:39:50 +02:00
Your Name
c3bb49d229 proxywatchd: make use of verifycert 2021-07-27 22:36:24 +02:00
Your Name
d4dd2a42ea proxywatchd.py: randomly choose from available regex keys 2021-07-05 16:54:17 +02:00
Your Name
d78212ac50 proxywatchd.py: add more websites 2021-07-05 16:45:02 +02:00
Your Name
a0bc4e4502 do not perform ssl_check when successive_success=0
aestetics
2021-07-03 20:11:46 +02:00
Your Name
f6c3347eb4 proxywatchd: match specific regex per server/protocol 2021-07-03 19:59:47 +02:00
Your Name
a005cc109c force ssl check (mitm) every 3 consecutive success check
needs a new entry in proxies.sqlite

sqlite3 proxies.sqlite "alter table proxylist add consecutive_success int"
sqlite3 proxies.sqlite "update proxylist set consecutive_success=0"
2021-07-03 18:47:34 +02:00
Your Name
11c5bd67b3 add http check 2021-07-03 18:47:34 +02:00
Your Name
6b6cd94cec spaces to tabs 2021-06-27 12:31:15 +02:00
Your Name
d3d83e1d90 changes 2021-05-12 08:06:03 +02:00
Your Name
9a7a913401 proxywatchd.py: also accept "ERROR" as success message (heh) 2021-05-02 16:11:57 +02:00
Your Name
cae6f75643 changs 2021-05-02 00:22:12 +02:00
Your Name
298987612c proxywatchd: do not test that much oldies 2021-02-10 22:25:41 +01:00
Your Name
9aa2c91f41 more random changes 2021-02-06 11:00:25 +01:00
Your Name
e15b9d2994 more changes 2021-02-04 23:06:37 +01:00
Your Name
78b29a1187 some changes 2021-01-24 03:52:56 +01:00
Mickaël Serneels
f179080cca use geoloc
now saves proxy's country in db
2019-05-17 22:59:32 +02:00
rofl0r
866f308322 proxywatchd: remove bogus blanket exception handler
this would catch *any* exception, including typos
2019-05-01 20:05:57 +01:00
Mickaël Serneels
d09244d04d proxywatchd: fix Exception error
Exception in thread Thread-9:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "proxywatchd.py", line 200, in workloop
    job.run()
  File "proxywatchd.py", line 123, in run
    sock, proto, duration, tor, srv, failinc = self.connect_socket()
ValueError: need more than 5 values to unpack
2019-05-01 17:43:28 +02:00
Mickaël Serneels
7aea9a3e53 irc: minimize possible response code 2019-05-01 17:43:28 +02:00
Mickaël Serneels
7b9f8b2e00 create socks4_resolve()
moves socks4 resolution out of socket_connect block
2019-05-01 17:43:28 +02:00
Mickaël Serneels
bad4d25bcf make watchd.tor_safeguard a configurable option (default: True) 2019-05-01 17:43:28 +02:00
Mickaël Serneels
67aec84320 fix Exception error
Exception in thread Thread-8:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "proxywatchd.py", line 191, in workloop
    job.run()
  File "proxywatchd.py", line 114, in run
    sock, proto, duration, tor, srv, failinc = self.connect_socket()
  File "proxywatchd.py", line 76, in connect_socket
    sock.send('%s\n' % random.choice(['NICK', 'USER', 'JOIN', 'MODE', 'PART', 'INVITE', 'KNOCK', 'WHOIS', 'WHO', 'NOTICE', 'PRIVMSG', 'PING', 'QUIT']))
  File "rocksock.py", line 279, in send
    return self.sock.sendall(buf)
  File "/usr/lib/python2.7/ssl.py", line 741, in sendall
    v = self.send(data[count:])
  File "/usr/lib/python2.7/ssl.py", line 707, in send
    v = self._sslobj.write(data)
error: [Errno 32] Broken pipe
2019-05-01 17:43:28 +02:00
mickael
310b01140a irc: implement use_ssl = 2
0: disabled, 1: enabled, 2: maybe
default is 0
2019-05-01 17:43:28 +02:00
mickael
1ad5ca53e5 take care of old proxies
test old proxies during free time
2019-03-05 22:29:16 +00:00
rofl0r
0734635e30 watchd main thread: be less nervous 2019-01-18 15:35:19 +00:00
rofl0r
ddee92d20f watchd: introduce configurable 'outage_threshold' 2019-01-18 15:34:49 +00:00
mickael
aaac14d34e worker: add threading lock
add lock to avoid same proxy to be scanned multiple time when
a small number a jobs is handed to worker
2019-01-13 16:50:54 +00:00
rofl0r
7d59404d31 watchd: add totals statistics 2019-01-11 00:52:11 +00:00
mickael
4c6a83373f split databases 2019-01-11 00:25:01 +00:00
rofl0r
ef9158015f proxywatchd: make checktime constants configurable
this requires only saving the last checked time in `tested`.
you can run the following sql statement to update the existing values
in the database:
sqlite3 proxylist.sqlite \
"update proxylist set tested=tested-(1800+(failed*3600)) where failed < 6"
2019-01-11 00:25:01 +00:00
rofl0r
befb346941 proxywatchd: preliminary support for ip caching
whenever we make a socks4 check, the ip of the destination server
needs to be resolved because socks4 does not support server-side
dns resolution. in order to prevent doing the same lookups over
and over, we know manually resolve the ip before first usage, and
store it in a cache.
2019-01-10 19:22:21 +00:00
rofl0r
bd1e85a883 proxywatchd: add safeguards against tor outage 2019-01-08 16:06:01 +00:00
rofl0r
fb917184b0 watchd: remove catch-it-all except statements
it took considerable time to figure out why it suddenly stopped
working due to a typo inside the try/except block.
2019-01-08 03:34:44 +00:00
rofl0r
f16f754b0e implement combo config parser
allows all options to be overridden by command line.

e.g.
[watchd]
threads=10
debug=false

--watch.threads=50 --debug=true
2019-01-08 02:17:10 +00:00
mickael
15bdbf63fd pyflakes: remove unused imports 2019-01-07 23:27:28 +00:00
mickael
1b3ce72efc add and use combining class 2019-01-07 23:19:14 +00:00
rofl0r
dfa90109da watchd: first halfbaked attempt to detect/combat tor issues 2019-01-07 21:35:36 +00:00
rofl0r
9e3c08fcf4 watchd: do not count name resolution errors as proxy fail 2019-01-07 15:57:15 +00:00
rofl0r
be99b8517b watchd: beautify logging 2019-01-07 15:38:51 +00:00
rofl0r
d93f14ad67 watchd: distribute remaining jobs among threads if one is idle 2019-01-07 15:38:51 +00:00
rofl0r
497f8b2050 watchd: remove non-printable chars from logged response 2019-01-07 02:54:33 +00:00
rofl0r
e51dbe113b watchd: optional debug output 2019-01-06 22:44:18 +00:00
rofl0r
312f77469e watchd: disable some noise in the proxy-found log output 2019-01-06 22:42:16 +00:00