Mickaël Serneels 0155c6f2ad ppf: check content-type (once) before trying to download/extract proxies ...

avoid trying to extract stuff from pdf and such (only accept text/*)

REQUIRES:
sqlite3 websites.sqlite "alter table uris add content_type text"

Don't test known uris:
sqlite3 websites.sqlite "update uris set content_type='text/manual' WHERE error=0"

2019-05-01 17:43:28 +02:00

comboparse.py

comboparse: fix bug with bool cmd args always True

2019-01-13 16:40:12 +00:00

config.ini.sample

watchd: introduce configurable 'outage_threshold'

2019-01-18 15:34:49 +00:00

config.py

oldies_multi: change default value from 100 to 10

2019-05-01 17:43:28 +02:00

dbs.py

set default error value to 1 for new urls

2019-05-01 17:43:28 +02:00

fetch.py

ppf: check content-type (once) before trying to download/extract proxies

2019-05-01 17:43:28 +02:00

http2.py

rocksock/http2: bump to latest

2019-01-08 15:48:38 +00:00

import.txt

update imports.txt

2019-05-01 17:43:28 +02:00

misc.py

pretty: hide printed '^C' when ^C'ing

2019-01-07 23:36:56 +00:00

mysqlite.py

mysqlite.py: improve

2019-01-18 20:42:15 +00:00

ppf.py

ppf: check content-type (once) before trying to download/extract proxies

2019-05-01 17:43:28 +02:00

proxywatchd.py

proxywatchd: fix Exception error

2019-05-01 17:43:28 +02:00

remove_bad_proxies_from_proxylist.bash

working but inefficient script to remove invalid proxies from db

2019-01-09 00:41:46 +00:00

rocksock.py

rocksock: bump to latest

2019-01-10 19:22:21 +00:00

scraper.py

search: more changes

2019-05-01 17:43:28 +02:00

search_terms.txt

initial commit

2019-01-03 16:36:31 +00:00

servers.txt

remove noexistent server

2019-03-05 22:29:16 +00:00

servers.txt.sample

add servers.txt.sample

2019-03-05 22:29:16 +00:00

soup_parser.py

move includes in root directory

2019-01-05 00:22:28 +00:00

urignore.txt

update urignore

2019-05-01 17:43:28 +02:00

Description

No description provided

3.7 MiB

Languages

Python 74.2%

JavaScript 10.5%

CSS 7.4%

Shell 4%

HTML 3.7%

Other 0.2%