From 4547ec318819ff2db6fce8f17d017242f52fc5a5 Mon Sep 17 00:00:00 2001 From: Username Date: Sat, 20 Dec 2025 18:25:55 +0100 Subject: [PATCH] roadmap: update completed work --- ROADMAP.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/ROADMAP.md b/ROADMAP.md index 8a4298d..06c2d38 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -211,6 +211,16 @@ PPF (Proxy Fetcher) is a Python 2 proxy scraping and validation framework design - [x] Fixed IP/port cleansing in ppf.py extract_proxies() - [x] Updated .gitignore, removed .pyc files +### Database Optimization (Done) +- [x] Enable SQLite WAL mode for better concurrency +- [x] Add indexes for common query patterns (failed, tested, proto, error, check_time) +- [x] Optimize batch inserts (remove redundant SELECT before INSERT OR IGNORE) + +### Dependency Reduction (Done) +- [x] Make lxml optional (removed from requirements) +- [x] Make IP2Location optional (graceful fallback) +- [x] Add --nobs flag for stdlib HTMLParser fallback (bs4 optional) + --- ## Technical Debt