docs: mark priority queue complete
This commit is contained in:
46
TODO.md
46
TODO.md
@@ -74,44 +74,16 @@ to fetch.py. Updated ppf.py to use these functions instead of local cache.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### [ ] 9. Priority Queue for Proxy Testing
|
### [x] 9. Priority Queue for Proxy Testing
|
||||||
|
|
||||||
**Problem:** All proxies tested with equal priority. Should prioritize:
|
**Completed.** Added priority-based job scheduling for proxy tests.
|
||||||
- Recently successful proxies (verify still working)
|
- PriorityJobQueue class with heap-based ordering
|
||||||
- New proxies (determine if usable)
|
- calculate_priority() assigns priority 0-4 based on proxy state
|
||||||
- Low fail-count proxies
|
- Priority 0: New proxies (never tested)
|
||||||
|
- Priority 1: Working proxies (no failures)
|
||||||
**Implementation:**
|
- Priority 2: Low fail count (< 3)
|
||||||
```python
|
- Priority 3-4: Medium/high fail count
|
||||||
# proxywatchd.py
|
- Integrated into prepare_jobs() for automatic prioritization
|
||||||
import heapq
|
|
||||||
|
|
||||||
class PriorityJobQueue:
|
|
||||||
"""Priority queue wrapper for proxy test jobs."""
|
|
||||||
def __init__(self):
|
|
||||||
self.heap = []
|
|
||||||
self.lock = threading.Lock()
|
|
||||||
|
|
||||||
def put(self, job, priority):
|
|
||||||
"""Lower priority number = higher priority."""
|
|
||||||
with self.lock:
|
|
||||||
heapq.heappush(self.heap, (priority, id(job), job))
|
|
||||||
|
|
||||||
def get(self, timeout=None):
|
|
||||||
"""Get highest priority job."""
|
|
||||||
# ... implementation with timeout
|
|
||||||
```
|
|
||||||
|
|
||||||
Priority calculation:
|
|
||||||
- New proxy (retrievals=0): priority 0
|
|
||||||
- Recent success (last_success < 1hr): priority 1
|
|
||||||
- Low fail count (failed < 3): priority 2
|
|
||||||
- Medium fail count: priority 3
|
|
||||||
- High fail count: priority 4
|
|
||||||
|
|
||||||
**Files:** proxywatchd.py
|
|
||||||
**Effort:** Medium
|
|
||||||
**Risk:** Medium
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user