mickael
ecc6bbce10
spaces2tab
2019-01-06 01:18:08 +00:00
mickael
922b630881
print number of defined threads
2019-01-06 00:51:20 +00:00
mickael
84a1de26c3
sqlite: do not create tables with "duration" column
2019-01-06 00:50:35 +00:00
mickael
4d513898fb
make "submit_after" a configuratble option (config.ini)
2019-01-06 00:38:22 +00:00
mickael
1cafce67f0
no need to store 'duration' value in db
2019-01-05 23:55:40 +00:00
mickael
67cf6a18ea
get rid of self.duration
2019-01-05 23:40:46 +00:00
mickael
dd4473279a
fix: store total_duration in ms
2019-01-05 23:39:36 +00:00
mickael
0cb3525fe9
store total_duration in ms
2019-01-05 23:23:21 +00:00
mickael
aab72ad68b
Merge branch 'master' of https://gitlab.com/mserneels/ppf
2019-01-05 22:57:57 +00:00
mserneels
5d64d07d00
Merge branch 'changes3' into 'master'
...
proxywatchd: improve logging
See merge request mserneels/ppf!3
2019-01-05 22:55:04 +00:00
mickael
d93f4dcaf2
introduce success_count and total_duration (proxylist.sqlit
...
run those commands to update the database:
sqlite3 proxylist.sqlite "alter table proxylist add success_count int"
sqlite3 proxylist.sqlite "alter table proxylist add total_duration int"
sqlite3 proxylist.sqlite "update proxylist set success_count=0,total_duration=0"
2019-01-05 22:24:38 +00:00
rofl0r
2bb75b6c42
proxywatchd: improve logging
2019-01-05 22:19:49 +00:00
mickael
3348435e21
remove unused variables
2019-01-05 21:56:45 +00:00
mserneels
b071400933
Merge branch 'changes2' into 'master'
...
Changes2
See merge request mserneels/ppf!2
2019-01-05 18:38:39 +00:00
rofl0r
dacbc8177a
proxywatchd: fix memleak
2019-01-05 18:31:36 +00:00
rofl0r
74d9d965bb
proxywatchd: always use a new mysql obj
...
could prevent memleaks, also helps thread-ownership issues.
2019-01-05 18:14:52 +00:00
mickael
cb342c3818
sqlite: use executemany
...
avoid loop inserting the whole list in a single query
2019-01-05 17:42:57 +00:00
mserneels
f79a537279
Merge branch 'changes' into 'master'
...
Changes
See merge request mserneels/ppf!1
2019-01-05 17:35:01 +00:00
rofl0r
af8f82924f
fix logic so threads do an orderly shutdown
...
basically the issue was that the main loop received the SIGINT
and therefore broke out before reaching the parts of the code
that care about bringing down the child threads.
therefore there's now a finish() method that needs to be called
after stop().
because sqlite dbs insists to be used from the thread that created
the object, the DB cleanup operation are done from the thread
that controls it.
for standalone operation, in order to keep the main thread alive,
an additional run() method is used. this is not necessary when
used via ppf.py.
2019-01-05 17:17:27 +00:00
rofl0r
bb3da7122e
ppf: properly reraise keyboard interrupts
2019-01-05 17:11:08 +00:00
rofl0r
47221bae94
properly clean submitted job list after db write
2019-01-05 16:44:31 +00:00
rofl0r
42f35855ea
use a variable for the min number of jobs to collect
...
this variable determines after how many results the db is written to.
with a low value, it is written often and could more lock issues
if 2 threads concur for the db.
2019-01-05 16:39:58 +00:00
rofl0r
f45cd1190c
fix performance issue in proxywatchd
2019-01-05 06:58:46 +00:00
rofl0r
9ac3ed45d6
rewrite threading code in jobwatchd
...
now it distributes the tasks properly among all threads,
and it can be used as a standalone program.
there are some minor performance issues which will be fixed shortly.
2019-01-05 06:35:41 +00:00
rofl0r
ff5d121d5f
allow some config options optionally to be set by command line
2019-01-05 04:41:31 +00:00
rofl0r
ffbe450aee
outsource configuration to external module
2019-01-05 03:47:03 +00:00
rofl0r
05294186d4
config sample: remove all unused items
2019-01-05 03:41:42 +00:00
mickael
09ea30b4e4
undo last change
2019-01-05 02:39:40 +00:00
mickael
f8b1b53fe9
cpu: be gentle
...
sleep 0.1 sec when adding proxies and urls to database
2019-01-05 01:17:12 +00:00
mickael
8f6f9d2f2e
fix nonsense
2019-01-05 01:12:27 +00:00
mickael
6df8fac96e
fix: don't re-test all protos every time
...
all protos (http, socks4, socks5) were tested even for working proxies (whose protocol has been saved..)
also, don't save dummy 'http' protocol for non working proxies..
2019-01-05 01:04:50 +00:00
mickael
85043d8e8a
move includes in root directory
2019-01-05 00:22:28 +00:00
mickael
df2a2ec072
sort server list; remove freenode
2019-01-04 23:59:06 +00:00
mickael
97416017c4
rename is_usable_ipv4 -> is_reserved_ipv4
2019-01-04 22:52:45 +00:00
mickael
5a2d994b49
get rid of ipcalc
2019-01-04 22:45:15 +00:00
mickael
0c271de1d7
tor: use socks4 instead of socks5
2019-01-04 21:58:30 +00:00
mickael
563f3323ab
remove hardcoded tor hosts
...
use user config
+ minor reorg.
2019-01-04 21:52:29 +00:00
mickael
4a60a23df3
fix imports
...
+ move soup_parser.py to includes
+ reorganize imports
+ remove double
2019-01-04 21:23:22 +00:00
mickael
4926d6c383
spaces2tabs
2019-01-04 21:23:22 +00:00
mickael
d7f79708ca
misc minor changes
...
proxywatchd: remove sleep
ppf: update proxies schemes
2019-01-04 20:23:50 +00:00
mickael
6453494839
bugfix: wrong 'next-check' value inserted in db
...
the "next check" value for working proxy were set to "time.time()", which (could have) caused the working proxies to be checked sooner than expected.
2019-01-04 20:09:05 +00:00
mickael
785b7c52a5
connect_socket: minor changes
...
small code rewrite and clarification
2019-01-04 00:18:51 +00:00
mickael
d8dd453bdf
add sample config file
2019-01-03 23:28:35 +00:00
mickael
f0b7e2dc2f
cleansing
2019-01-03 23:28:35 +00:00
mickael
d7eaa62ae8
test-irc: use SSL (6697) by default
2019-01-03 20:54:04 +00:00
mickael
3fb2dc40a6
initial commit
2019-01-03 16:36:31 +00:00
