username/ppf
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 10d6b3afd8 servers.txt: remove dead server rofl0r 2019-01-09 22:31:24 +00:00
  • 383ae6f431 fix: no uris were tested because commented" mickael 2019-01-10 00:21:57 +00:00
  • 0cbc434883 Merge branch 'experiment' into 'master' mserneels 2019-01-09 23:40:54 +00:00
  • da4f228479 discard urls who fail at first test mickael 2019-01-09 23:38:59 +00:00
  • 15dee0cd73 add -intitle:pdf to searx query mickael 2019-01-09 23:30:55 +00:00
  • e94644a60e searx: loop for 10 pages on each searx instance mickael 2019-01-09 22:55:55 +00:00
  • 8993727f03 changed regex mickael 2019-01-09 20:07:28 +00:00
  • 33887385f0 is_usable_proxy: group the 2 firsts lines mickael 2019-01-09 19:23:09 +00:00
  • 9828db79d4 is_usable_proxy(): dont check twice if A < 1 mickael 2019-01-09 19:11:05 +00:00
  • 6f0d5c1ffa modify and rename should_i_... function mickael 2019-01-09 19:01:55 +00:00
  • a74d6dfce8 do not save invalid IPs mickael 2019-01-09 00:42:28 +00:00
  • bde24ce0e9 working but inefficient script to remove invalid proxies from db mickael 2019-01-09 00:41:46 +00:00
  • 484fe325f8 Merge branch 'changes9' into 'master' mserneels 2019-01-08 19:55:45 +00:00
  • bd1e85a883 proxywatchd: add safeguards against tor outage rofl0r 2019-01-08 16:06:01 +00:00
  • 6e4c45175e ppf: add safeguards against tor outage rofl0r 2019-01-08 15:43:03 +00:00
  • b6839eea79 rocksock/http2: bump to latest rofl0r 2019-01-08 15:42:03 +00:00
  • b88a8426dc rocksock: bump to latest rofl0r 2019-01-08 04:38:28 +00:00
  • b9058b0334 http2: bump to latest rofl0r 2019-01-08 04:38:14 +00:00
  • 1f3179de48 ppf: check for valid ports rofl0r 2019-01-08 04:30:50 +00:00
  • 9ccf8b7854 ppf: write dates as int rofl0r 2019-01-08 04:18:25 +00:00
  • 35421b980e rocksock: bump to latest rofl0r 2019-01-08 03:35:40 +00:00
  • fb917184b0 watchd: remove catch-it-all except statements rofl0r 2019-01-08 03:34:41 +00:00
  • 38d89f5bd9 ppf: add option for number of http retries rofl0r 2019-01-08 03:30:31 +00:00
  • 115c4a56f5 ppf: honor timeout rofl0r 2019-01-08 03:25:52 +00:00
  • f16f754b0e implement combo config parser rofl0r 2019-01-08 02:17:04 +00:00
  • e7b8d526c0 ppf: print url if fetching failed rofl0r 2019-01-08 00:46:41 +00:00
  • 20bc8be2fa pretty: hide printed '^C' when ^C'ing mickael 2019-01-07 23:36:56 +00:00
  • 15bdbf63fd pyflakes: remove unused imports mickael 2019-01-07 23:27:28 +00:00
  • 1b3ce72efc add and use combining class mickael 2019-01-07 23:19:14 +00:00
  • e31ac03f6f avoid double "checktime" usage mickael 2019-01-07 22:09:22 +00:00
  • 6c8b707637 Merge branch 'master' of ssh://gitlab.com/mserneels/ppf mickael 2019-01-07 21:41:46 +00:00
  • 1288dca38f fixme: change var names mickael 2019-01-07 21:41:41 +00:00
  • a86cc925be Merge branch 'changes8' into 'master' mserneels 2019-01-07 21:38:12 +00:00
  • dfa90109da watchd: first halfbaked attempt to detect/combat tor issues rofl0r 2019-01-07 21:35:36 +00:00
  • aeff09d2b3 move math function inside the sql statement mickael 2019-01-07 21:11:08 +00:00
  • f47745c8d6 Merge branch 'changes7' into 'master' mserneels 2019-01-07 18:56:23 +00:00
  • 9d41e21258 remove dead irc servers (DNS NXDOMAIN) rofl0r 2019-01-07 16:22:26 +00:00
  • 9e8beca4e1 do not treat empty lines as servers rofl0r 2019-01-07 16:22:06 +00:00
  • 9e3c08fcf4 watchd: do not count name resolution errors as proxy fail rofl0r 2019-01-07 15:57:15 +00:00
  • be99b8517b watchd: beautify logging rofl0r 2019-01-07 05:45:09 +00:00
  • d93f14ad67 watchd: distribute remaining jobs among threads if one is idle rofl0r 2019-01-07 05:39:43 +00:00
  • 898c8f36ee ppf: fix cpu hogs rofl0r 2019-01-07 05:38:35 +00:00
  • ad7c7fce67 ppf: use timeout and only 1 try for http rofl0r 2019-01-07 05:37:44 +00:00
  • 497f8b2050 watchd: remove non-printable chars from logged response rofl0r 2019-01-07 02:54:33 +00:00
  • c8d93751fe rocksock: bump to latest rofl0r 2019-01-07 02:53:36 +00:00
  • 8b15faf84d ppf: change user-agent; use headers mickael 2019-01-06 23:29:30 +00:00
  • 1fd4792218 Merge branch 'merge1' into 'master' mserneels 2019-01-06 22:45:50 +00:00
  • e51dbe113b watchd: optional debug output rofl0r 2019-01-06 22:44:18 +00:00
  • 312f77469e watchd: disable some noise in the proxy-found log output rofl0r 2019-01-06 22:34:29 +00:00
  • 5a6c49d88b watchd: collect stats rofl0r 2019-01-06 22:19:43 +00:00
  • 3223cc82c4 use http2.py instead of requests mickael 2019-01-06 22:22:42 +00:00
  • 62153a6972 bump to latest mickael 2019-01-06 21:34:51 +00:00
  • 358044ccea watchd: make use of ssl configurable mickael 2019-01-06 20:09:27 +00:00
  • 0ab3a6d066 connect_socket: fix proto bug mickael 2019-01-06 19:43:05 +00:00
  • 91654d4b9e remove unused "is_drone_bl()" function mickael 2019-01-06 19:12:42 +00:00
  • 1a025f102f only load search/bad terms when "search" arg is enabled mickael 2019-01-06 18:31:42 +00:00
  • 5e9f8baf56 remove unused imports mickael 2019-01-06 18:27:06 +00:00
  • e4636df468 delay thread startup mickael 2019-01-06 17:52:56 +00:00
  • 03a1c47ff2 Merge branch 'changes5' into 'master' mserneels 2019-01-06 16:38:44 +00:00
  • 7d8583f7cc no need to re-raise keyboardinterrupt rofl0r 2019-01-06 16:30:32 +00:00
  • 14e4b832f2 proxywatchd: dont delete timestamp when printing halt msg rofl0r 2019-01-06 16:29:17 +00:00
  • 6c515d9335 don't continue testing offline proxies rofl0r 2019-01-06 16:12:50 +00:00
  • 41ce83a963 rocksock.py: bump to latest mickael 2019-01-06 16:05:48 +00:00
  • 114c77dcf8 Merge branch 'changes4' into 'master' mserneels 2019-01-06 15:40:01 +00:00
  • a0dcd9fa8f save total_duration as int into the db rofl0r 2019-01-06 15:23:23 +00:00
  • 1adf78119d take care of "failinc" mickael 2019-01-06 14:57:09 +00:00
  • cff3e1304c remove rocksockexception block mickael 2019-01-06 14:55:31 +00:00
  • 0a34833d3d "prettier" thread exit message mickael 2019-01-06 05:13:23 +00:00
  • 6b4ca67b33 use e.get_failedproxy() mickael 2019-01-06 05:02:02 +00:00
  • 1f33a632fb update rocksock.py to latest mickael 2019-01-06 04:53:20 +00:00
  • c661047381 connect_socket: single srv for all protos mickael 2019-01-06 04:42:17 +00:00
  • e872caaa77 catch some rocksock errors mickael 2019-01-06 04:12:57 +00:00
  • 64d9da9156 sleep even when no proxies are added mickael 2019-01-06 02:58:58 +00:00
  • 63b77043ac minor changes mickael 2019-01-06 01:35:18 +00:00
  • aab7ef15d1 remove unnecessary global variable mickael 2019-01-06 01:29:35 +00:00
  • ecc6bbce10 spaces2tab mickael 2019-01-06 01:18:08 +00:00
  • 922b630881 print number of defined threads mickael 2019-01-06 00:51:20 +00:00
  • 84a1de26c3 sqlite: do not create tables with "duration" column mickael 2019-01-06 00:50:35 +00:00
  • 4d513898fb make "submit_after" a configuratble option (config.ini) mickael 2019-01-06 00:38:22 +00:00
  • 1cafce67f0 no need to store 'duration' value in db mickael 2019-01-05 23:55:40 +00:00
  • 67cf6a18ea get rid of self.duration mickael 2019-01-05 23:40:46 +00:00
  • dd4473279a fix: store total_duration in ms mickael 2019-01-05 23:39:36 +00:00
  • 0cb3525fe9 store total_duration in ms mickael 2019-01-05 23:23:21 +00:00
  • aab72ad68b Merge branch 'master' of https://gitlab.com/mserneels/ppf mickael 2019-01-05 22:57:57 +00:00
  • 5d64d07d00 Merge branch 'changes3' into 'master' mserneels 2019-01-05 22:55:04 +00:00
  • d93f4dcaf2 introduce success_count and total_duration (proxylist.sqlit mickael 2019-01-05 22:24:38 +00:00
  • 2bb75b6c42 proxywatchd: improve logging rofl0r 2019-01-05 22:19:49 +00:00
  • 3348435e21 remove unused variables mickael 2019-01-05 21:56:45 +00:00
  • b071400933 Merge branch 'changes2' into 'master' mserneels 2019-01-05 18:38:39 +00:00
  • dacbc8177a proxywatchd: fix memleak rofl0r 2019-01-05 18:31:36 +00:00
  • 74d9d965bb proxywatchd: always use a new mysql obj rofl0r 2019-01-05 18:14:41 +00:00
  • cb342c3818 sqlite: use executemany mickael 2019-01-05 17:42:57 +00:00
  • f79a537279 Merge branch 'changes' into 'master' mserneels 2019-01-05 17:35:01 +00:00
  • af8f82924f fix logic so threads do an orderly shutdown rofl0r 2019-01-05 17:13:39 +00:00
  • bb3da7122e ppf: properly reraise keyboard interrupts rofl0r 2019-01-05 17:11:08 +00:00
  • 47221bae94 properly clean submitted job list after db write rofl0r 2019-01-05 16:44:31 +00:00
  • 42f35855ea use a variable for the min number of jobs to collect rofl0r 2019-01-05 16:39:55 +00:00
  • f45cd1190c fix performance issue in proxywatchd rofl0r 2019-01-05 06:58:46 +00:00
  • 9ac3ed45d6 rewrite threading code in jobwatchd rofl0r 2019-01-05 06:35:41 +00:00
  • ff5d121d5f allow some config options optionally to be set by command line rofl0r 2019-01-05 04:41:25 +00:00