Commit Graph

78 Commits

Author SHA1 Message Date
Your Name
a0bc4e4502 do not perform ssl_check when successive_success=0
aestetics
2021-07-03 20:11:46 +02:00
Your Name
f6c3347eb4 proxywatchd: match specific regex per server/protocol 2021-07-03 19:59:47 +02:00
Your Name
a005cc109c force ssl check (mitm) every 3 consecutive success check
needs a new entry in proxies.sqlite

sqlite3 proxies.sqlite "alter table proxylist add consecutive_success int"
sqlite3 proxies.sqlite "update proxylist set consecutive_success=0"
2021-07-03 18:47:34 +02:00
Your Name
11c5bd67b3 add http check 2021-07-03 18:47:34 +02:00
Your Name
6b6cd94cec spaces to tabs 2021-06-27 12:31:15 +02:00
Your Name
d3d83e1d90 changes 2021-05-12 08:06:03 +02:00
Your Name
9a7a913401 proxywatchd.py: also accept "ERROR" as success message (heh) 2021-05-02 16:11:57 +02:00
Your Name
cae6f75643 changs 2021-05-02 00:22:12 +02:00
Your Name
298987612c proxywatchd: do not test that much oldies 2021-02-10 22:25:41 +01:00
Your Name
9aa2c91f41 more random changes 2021-02-06 11:00:25 +01:00
Your Name
e15b9d2994 more changes 2021-02-04 23:06:37 +01:00
Your Name
78b29a1187 some changes 2021-01-24 03:52:56 +01:00
Mickaël Serneels
f179080cca use geoloc
now saves proxy's country in db
2019-05-17 22:59:32 +02:00
rofl0r
866f308322 proxywatchd: remove bogus blanket exception handler
this would catch *any* exception, including typos
2019-05-01 20:05:57 +01:00
Mickaël Serneels
d09244d04d proxywatchd: fix Exception error
Exception in thread Thread-9:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "proxywatchd.py", line 200, in workloop
    job.run()
  File "proxywatchd.py", line 123, in run
    sock, proto, duration, tor, srv, failinc = self.connect_socket()
ValueError: need more than 5 values to unpack
2019-05-01 17:43:28 +02:00
Mickaël Serneels
7aea9a3e53 irc: minimize possible response code 2019-05-01 17:43:28 +02:00
Mickaël Serneels
7b9f8b2e00 create socks4_resolve()
moves socks4 resolution out of socket_connect block
2019-05-01 17:43:28 +02:00
Mickaël Serneels
bad4d25bcf make watchd.tor_safeguard a configurable option (default: True) 2019-05-01 17:43:28 +02:00
Mickaël Serneels
67aec84320 fix Exception error
Exception in thread Thread-8:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "proxywatchd.py", line 191, in workloop
    job.run()
  File "proxywatchd.py", line 114, in run
    sock, proto, duration, tor, srv, failinc = self.connect_socket()
  File "proxywatchd.py", line 76, in connect_socket
    sock.send('%s\n' % random.choice(['NICK', 'USER', 'JOIN', 'MODE', 'PART', 'INVITE', 'KNOCK', 'WHOIS', 'WHO', 'NOTICE', 'PRIVMSG', 'PING', 'QUIT']))
  File "rocksock.py", line 279, in send
    return self.sock.sendall(buf)
  File "/usr/lib/python2.7/ssl.py", line 741, in sendall
    v = self.send(data[count:])
  File "/usr/lib/python2.7/ssl.py", line 707, in send
    v = self._sslobj.write(data)
error: [Errno 32] Broken pipe
2019-05-01 17:43:28 +02:00
mickael
310b01140a irc: implement use_ssl = 2
0: disabled, 1: enabled, 2: maybe
default is 0
2019-05-01 17:43:28 +02:00
mickael
1ad5ca53e5 take care of old proxies
test old proxies during free time
2019-03-05 22:29:16 +00:00
rofl0r
0734635e30 watchd main thread: be less nervous 2019-01-18 15:35:19 +00:00
rofl0r
ddee92d20f watchd: introduce configurable 'outage_threshold' 2019-01-18 15:34:49 +00:00
mickael
aaac14d34e worker: add threading lock
add lock to avoid same proxy to be scanned multiple time when
a small number a jobs is handed to worker
2019-01-13 16:50:54 +00:00
rofl0r
7d59404d31 watchd: add totals statistics 2019-01-11 00:52:11 +00:00
mickael
4c6a83373f split databases 2019-01-11 00:25:01 +00:00
rofl0r
ef9158015f proxywatchd: make checktime constants configurable
this requires only saving the last checked time in `tested`.
you can run the following sql statement to update the existing values
in the database:
sqlite3 proxylist.sqlite \
"update proxylist set tested=tested-(1800+(failed*3600)) where failed < 6"
2019-01-11 00:25:01 +00:00
rofl0r
befb346941 proxywatchd: preliminary support for ip caching
whenever we make a socks4 check, the ip of the destination server
needs to be resolved because socks4 does not support server-side
dns resolution. in order to prevent doing the same lookups over
and over, we know manually resolve the ip before first usage, and
store it in a cache.
2019-01-10 19:22:21 +00:00
rofl0r
bd1e85a883 proxywatchd: add safeguards against tor outage 2019-01-08 16:06:01 +00:00
rofl0r
fb917184b0 watchd: remove catch-it-all except statements
it took considerable time to figure out why it suddenly stopped
working due to a typo inside the try/except block.
2019-01-08 03:34:44 +00:00
rofl0r
f16f754b0e implement combo config parser
allows all options to be overridden by command line.

e.g.
[watchd]
threads=10
debug=false

--watch.threads=50 --debug=true
2019-01-08 02:17:10 +00:00
mickael
15bdbf63fd pyflakes: remove unused imports 2019-01-07 23:27:28 +00:00
mickael
1b3ce72efc add and use combining class 2019-01-07 23:19:14 +00:00
rofl0r
dfa90109da watchd: first halfbaked attempt to detect/combat tor issues 2019-01-07 21:35:36 +00:00
rofl0r
9e3c08fcf4 watchd: do not count name resolution errors as proxy fail 2019-01-07 15:57:15 +00:00
rofl0r
be99b8517b watchd: beautify logging 2019-01-07 15:38:51 +00:00
rofl0r
d93f14ad67 watchd: distribute remaining jobs among threads if one is idle 2019-01-07 15:38:51 +00:00
rofl0r
497f8b2050 watchd: remove non-printable chars from logged response 2019-01-07 02:54:33 +00:00
rofl0r
e51dbe113b watchd: optional debug output 2019-01-06 22:44:18 +00:00
rofl0r
312f77469e watchd: disable some noise in the proxy-found log output 2019-01-06 22:42:16 +00:00
rofl0r
5a6c49d88b watchd: collect stats 2019-01-06 22:42:09 +00:00
mickael
358044ccea watchd: make use of ssl configurable 2019-01-06 20:09:27 +00:00
mickael
0ab3a6d066 connect_socket: fix proto bug 2019-01-06 19:43:05 +00:00
mickael
91654d4b9e remove unused "is_drone_bl()" function 2019-01-06 19:12:42 +00:00
mickael
e4636df468 delay thread startup 2019-01-06 17:52:56 +00:00
rofl0r
7d8583f7cc no need to re-raise keyboardinterrupt 2019-01-06 16:30:32 +00:00
rofl0r
14e4b832f2 proxywatchd: dont delete timestamp when printing halt msg 2019-01-06 16:29:17 +00:00
rofl0r
6c515d9335 don't continue testing offline proxies 2019-01-06 16:20:29 +00:00
rofl0r
a0dcd9fa8f save total_duration as int into the db
without that, the DB saves it as float even though the column type is
INT.

requires a run of

sqlite3 proxylist.sqlite "update proxylist set success_count=0,total_duration=0"

to fix existing entries.
2019-01-06 15:23:23 +00:00
mickael
1adf78119d take care of "failinc" 2019-01-06 14:57:09 +00:00