Mickaël Serneels
67aec84320
fix Exception error
...
Exception in thread Thread-8:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "proxywatchd.py", line 191, in workloop
job.run()
File "proxywatchd.py", line 114, in run
sock, proto, duration, tor, srv, failinc = self.connect_socket()
File "proxywatchd.py", line 76, in connect_socket
sock.send('%s\n' % random.choice(['NICK', 'USER', 'JOIN', 'MODE', 'PART', 'INVITE', 'KNOCK', 'WHOIS', 'WHO', 'NOTICE', 'PRIVMSG', 'PING', 'QUIT']))
File "rocksock.py", line 279, in send
return self.sock.sendall(buf)
File "/usr/lib/python2.7/ssl.py", line 741, in sendall
v = self.send(data[count:])
File "/usr/lib/python2.7/ssl.py", line 707, in send
v = self._sslobj.write(data)
error: [Errno 32] Broken pipe
2019-05-01 17:43:28 +02:00
mickael
310b01140a
irc: implement use_ssl = 2
...
0: disabled, 1: enabled, 2: maybe
default is 0
2019-05-01 17:43:28 +02:00
mickael
1ad5ca53e5
take care of old proxies
...
test old proxies during free time
2019-03-05 22:29:16 +00:00
rofl0r
0734635e30
watchd main thread: be less nervous
2019-01-18 15:35:19 +00:00
rofl0r
ddee92d20f
watchd: introduce configurable 'outage_threshold'
2019-01-18 15:34:49 +00:00
mickael
aaac14d34e
worker: add threading lock
...
add lock to avoid same proxy to be scanned multiple time when
a small number a jobs is handed to worker
2019-01-13 16:50:54 +00:00
rofl0r
7d59404d31
watchd: add totals statistics
2019-01-11 00:52:11 +00:00
mickael
4c6a83373f
split databases
2019-01-11 00:25:01 +00:00
rofl0r
ef9158015f
proxywatchd: make checktime constants configurable
...
this requires only saving the last checked time in `tested`.
you can run the following sql statement to update the existing values
in the database:
sqlite3 proxylist.sqlite \
"update proxylist set tested=tested-(1800+(failed*3600)) where failed < 6"
2019-01-11 00:25:01 +00:00
rofl0r
befb346941
proxywatchd: preliminary support for ip caching
...
whenever we make a socks4 check, the ip of the destination server
needs to be resolved because socks4 does not support server-side
dns resolution. in order to prevent doing the same lookups over
and over, we know manually resolve the ip before first usage, and
store it in a cache.
2019-01-10 19:22:21 +00:00
rofl0r
bd1e85a883
proxywatchd: add safeguards against tor outage
2019-01-08 16:06:01 +00:00
rofl0r
fb917184b0
watchd: remove catch-it-all except statements
...
it took considerable time to figure out why it suddenly stopped
working due to a typo inside the try/except block.
2019-01-08 03:34:44 +00:00
rofl0r
f16f754b0e
implement combo config parser
...
allows all options to be overridden by command line.
e.g.
[watchd]
threads=10
debug=false
--watch.threads=50 --debug=true
2019-01-08 02:17:10 +00:00
mickael
15bdbf63fd
pyflakes: remove unused imports
2019-01-07 23:27:28 +00:00
mickael
1b3ce72efc
add and use combining class
2019-01-07 23:19:14 +00:00
rofl0r
dfa90109da
watchd: first halfbaked attempt to detect/combat tor issues
2019-01-07 21:35:36 +00:00
rofl0r
9e3c08fcf4
watchd: do not count name resolution errors as proxy fail
2019-01-07 15:57:15 +00:00
rofl0r
be99b8517b
watchd: beautify logging
2019-01-07 15:38:51 +00:00
rofl0r
d93f14ad67
watchd: distribute remaining jobs among threads if one is idle
2019-01-07 15:38:51 +00:00
rofl0r
497f8b2050
watchd: remove non-printable chars from logged response
2019-01-07 02:54:33 +00:00
rofl0r
e51dbe113b
watchd: optional debug output
2019-01-06 22:44:18 +00:00
rofl0r
312f77469e
watchd: disable some noise in the proxy-found log output
2019-01-06 22:42:16 +00:00
rofl0r
5a6c49d88b
watchd: collect stats
2019-01-06 22:42:09 +00:00
mickael
358044ccea
watchd: make use of ssl configurable
2019-01-06 20:09:27 +00:00
mickael
0ab3a6d066
connect_socket: fix proto bug
2019-01-06 19:43:05 +00:00
mickael
91654d4b9e
remove unused "is_drone_bl()" function
2019-01-06 19:12:42 +00:00
mickael
e4636df468
delay thread startup
2019-01-06 17:52:56 +00:00
rofl0r
7d8583f7cc
no need to re-raise keyboardinterrupt
2019-01-06 16:30:32 +00:00
rofl0r
14e4b832f2
proxywatchd: dont delete timestamp when printing halt msg
2019-01-06 16:29:17 +00:00
rofl0r
6c515d9335
don't continue testing offline proxies
2019-01-06 16:20:29 +00:00
rofl0r
a0dcd9fa8f
save total_duration as int into the db
...
without that, the DB saves it as float even though the column type is
INT.
requires a run of
sqlite3 proxylist.sqlite "update proxylist set success_count=0,total_duration=0"
to fix existing entries.
2019-01-06 15:23:23 +00:00
mickael
1adf78119d
take care of "failinc"
2019-01-06 14:57:09 +00:00
mickael
cff3e1304c
remove rocksockexception block
2019-01-06 14:55:31 +00:00
mickael
0a34833d3d
"prettier" thread exit message
2019-01-06 05:13:23 +00:00
mickael
6b4ca67b33
use e.get_failedproxy()
2019-01-06 05:02:02 +00:00
mickael
c661047381
connect_socket: single srv for all protos
2019-01-06 04:42:17 +00:00
mickael
e872caaa77
catch some rocksock errors
...
this allows ppf to skip checks when a theorical proxy fails in certain manners
2019-01-06 04:12:57 +00:00
mickael
aab7ef15d1
remove unnecessary global variable
2019-01-06 01:29:35 +00:00
mickael
ecc6bbce10
spaces2tab
2019-01-06 01:18:08 +00:00
mickael
922b630881
print number of defined threads
2019-01-06 00:51:20 +00:00
mickael
4d513898fb
make "submit_after" a configuratble option (config.ini)
2019-01-06 00:38:22 +00:00
mickael
1cafce67f0
no need to store 'duration' value in db
2019-01-05 23:55:40 +00:00
mickael
67cf6a18ea
get rid of self.duration
2019-01-05 23:40:46 +00:00
mickael
dd4473279a
fix: store total_duration in ms
2019-01-05 23:39:36 +00:00
mickael
0cb3525fe9
store total_duration in ms
2019-01-05 23:23:21 +00:00
mickael
aab72ad68b
Merge branch 'master' of https://gitlab.com/mserneels/ppf
2019-01-05 22:57:57 +00:00
mickael
d93f4dcaf2
introduce success_count and total_duration (proxylist.sqlit
...
run those commands to update the database:
sqlite3 proxylist.sqlite "alter table proxylist add success_count int"
sqlite3 proxylist.sqlite "alter table proxylist add total_duration int"
sqlite3 proxylist.sqlite "update proxylist set success_count=0,total_duration=0"
2019-01-05 22:24:38 +00:00
rofl0r
2bb75b6c42
proxywatchd: improve logging
2019-01-05 22:19:49 +00:00
mickael
3348435e21
remove unused variables
2019-01-05 21:56:45 +00:00
rofl0r
dacbc8177a
proxywatchd: fix memleak
2019-01-05 18:31:36 +00:00
