docs: update for v1.4.0 features

- Add anti-flood, rate limiting, scheduled cleanup to feature lists - Update version to 1.4.0, test count to 205 - Document /pastes endpoint with query parameters - Add anti-flood fields to /challenge response - Update CLI docs with new commands (list, search, export) - Add decision log entries for recent features
2025-12-20 21:36:09 +01:00
parent 98bc656c87
commit b47c26dd14
6 changed files with 129 additions and 12 deletions
--- a/PROJECT.md
+++ b/PROJECT.md
@@ -119,7 +119,7 @@ A self-hosted pastebin API that:

 ## Current Status

-**Version:** 1.2.0
+**Version:** 1.4.0

 ```
 ┌─────────────────────────────────┬────────────────────────────────────────────┐
@@ -134,6 +134,8 @@ A self-hosted pastebin API that:
 │ Paste expiry                    │ Complete
 │ Content-hash deduplication      │ Complete
 │ Proof-of-work                   │ Complete
+│ Anti-flood (dynamic PoW)        │ Complete
+│ IP-based rate limiting          │ Complete
 │ URL prefix support              │ Complete
 │ /client endpoint                │ Complete
 │ E2E encryption (CLI)            │ Complete
@@ -145,6 +147,8 @@ A self-hosted pastebin API that:
 │ Container deployment            │ Complete
 │ Security tooling                │ Complete
 │ CI/CD pipeline                  │ Complete
-│ Test suite                      │ 147 tests passing
+│ Scheduled cleanup               │ Complete
+│ CLI paste listing/search        │ Complete
+│ Test suite                      │ 205 tests passing
 └─────────────────────────────────┴────────────────────────────────────────────┘
 ```
--- a/README.md
+++ b/README.md
@@ -13,6 +13,8 @@ A lightweight, secure pastebin REST API built with Flask.
 - **Abuse prevention** - Content-hash deduplication throttles repeated identical submissions
 - **Entropy enforcement** - Optional minimum entropy requirement to enforce client-side encryption
 - **Proof-of-work** - Configurable computational puzzle prevents automated spam
+- **Anti-flood** - Dynamic PoW difficulty increases under attack, decays when abuse stops
+- **Rate limiting** - Per-IP request throttling with auth user multiplier
 - **E2E encryption** - Client-side AES-256-GCM encryption with key in URL fragment (zero-knowledge)
 - **Burn-after-read** - Single-access pastes that auto-delete after first retrieval
 - **Custom expiry** - Per-paste expiry override via X-Expiry header
@@ -229,6 +231,16 @@ Configuration via environment variables:
 | `FLASKPASTE_POW_DIFFICULTY` | `20` | PoW difficulty (leading zero bits, 0=disabled) |
 | `FLASKPASTE_POW_TTL` | `300` (5 min) | PoW challenge validity period |
 | `FLASKPASTE_POW_SECRET` | (auto) | Secret for signing PoW challenges |
+| `FLASKPASTE_ANTIFLOOD` | `1` | Enable anti-flood (dynamic PoW difficulty) |
+| `FLASKPASTE_ANTIFLOOD_WINDOW` | `60` | Anti-flood measurement window (seconds) |
+| `FLASKPASTE_ANTIFLOOD_THRESHOLD` | `5` | Requests per window before difficulty increase |
+| `FLASKPASTE_ANTIFLOOD_STEP` | `2` | Difficulty bits added per threshold breach |
+| `FLASKPASTE_ANTIFLOOD_MAX` | `28` | Maximum PoW difficulty |
+| `FLASKPASTE_ANTIFLOOD_DECAY` | `60` | Seconds before difficulty decreases |
+| `FLASKPASTE_RATE_LIMIT` | `1` | Enable IP-based rate limiting |
+| `FLASKPASTE_RATE_WINDOW` | `60` | Rate limit window (seconds) |
+| `FLASKPASTE_RATE_MAX` | `10` | Max requests per window (anon) |
+| `FLASKPASTE_RATE_AUTH_MULT` | `5` | Multiplier for authenticated users |
 | `FLASKPASTE_URL_PREFIX` | (empty) | URL prefix for reverse proxy deployments |
 | `FLASKPASTE_MIN_ENTROPY` | `0` | Min entropy bits/byte (0=disabled, 6.0=require encryption) |
 | `FLASKPASTE_MIN_ENTROPY_SIZE` | `256` | Only check entropy for content >= this size |
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -2,7 +2,7 @@

 ## Current State

-FlaskPaste v1.2.0 is deployed with PKI integration and comprehensive security tooling.
+FlaskPaste v1.4.0 is deployed with comprehensive security hardening and abuse prevention.

 **Implemented:**
 - Full REST API (CRUD operations)
@@ -11,6 +11,8 @@ FlaskPaste v1.2.0 is deployed with PKI integration and comprehensive security to
 - Minimal PKI (CA generation, certificate issuance, revocation)
 - Content-hash deduplication (abuse prevention)
 - Proof-of-work spam prevention
+- Anti-flood system (dynamic PoW difficulty under load)
+- IP-based rate limiting (configurable per-IP limits)
 - Entropy enforcement (require encrypted uploads)
 - E2E encryption in CLI (AES-256-GCM, key in URL fragment)
 - URL prefix support for reverse proxy deployments
@@ -18,11 +20,13 @@ FlaskPaste v1.2.0 is deployed with PKI integration and comprehensive security to
 - Automatic paste expiry
 - Burn-after-read pastes
 - Custom expiry per paste
+- Scheduled cleanup (pastes, hashes, rate limits)
 - Security headers and request tracing
 - Container deployment support
 - Security tooling (ruff, bandit, mypy, pip-audit)
 - CI/CD pipeline with lint, security, and test jobs
- Comprehensive test suite (147 tests)
+- CLI with list, search, update, export commands
+- Comprehensive test suite (205 tests)

 ## Phase 1: Hardening (Complete)

@@ -38,7 +42,7 @@ Focus: Production readiness and operational excellence.
 │ 4 │ Proxy trust validation          │ Done
 │ 5 │ Proof-of-work spam prevention   │ Done
 │ 6 │ Entropy enforcement             │ Done
-│ 7 │ Test coverage > 90%             │ Done (147 tests)
+│ 7 │ Test coverage > 90%             │ Done (205 tests)
 │ 8 │ Documentation complete          │ Done
 └───┴─────────────────────────────────┴────────────────────────────────────┘
 ```
@@ -67,14 +71,26 @@ Focus: User-requested enhancements within scope.
 ┌───┬─────────────────────────────────┬────────────────────────────────────┐
 │ # │ Feature                         │ Status
 ├───┼─────────────────────────────────┼────────────────────────────────────┤
-│ 1 │ E2E encryption (client-side)    │ Done (CLI -e flag, zero-knowledge)
+│ 1 │ E2E encryption (client-side)    │ Done (CLI encrypts by default)
 │ 2 │ URL prefix support              │ Done
 │ 3 │ Custom expiry per paste         │ Done (X-Expiry header)
 │ 4 │ Burn-after-read option          │ Done (X-Burn-After-Read header)
 │ 5 │ Minimal PKI (CA + issuance)     │ Done
+│ 6 │ Anti-flood (dynamic PoW)        │ Done (v1.4.0)
+│ 7 │ IP-based rate limiting          │ Done (v1.4.0)
+│ 8 │ Scheduled cleanup               │ Done (v1.4.0)
 └───┴─────────────────────────────────┴────────────────────────────────────┘
 ```

+### Anti-Flood System (v1.4.0)
+
+Dynamic proof-of-work difficulty that increases under abuse:
+- Base difficulty: 20 bits (configurable)
+- Threshold: 5 requests per 60s window triggers increase
+- Step: +2 bits per threshold breach
+- Maximum: 28 bits
+- Decay: Returns to base after 60s of normal traffic
+
 ### PKI Features

 Integrated certificate authority for mTLS:
@@ -103,15 +119,20 @@ Focus: Integration with external systems.

 ### CLI Client (Complete)

-Standalone Python CLI with encryption and PKI support:
- `fpaste create file.txt` - Create paste from file
- `fpaste create -e file.txt` - Create encrypted paste (E2E)
+Standalone Python CLI with encryption, PKI, and paste management:
+- `fpaste file.txt` - Create encrypted paste (file path shortcut)
+- `fpaste create -E file.txt` - Create unencrypted paste
 - `fpaste get <id>` - Get paste (auto-decrypts with URL fragment key)
 - `fpaste delete <id>` - Delete paste
- `fpaste info` - Show server info
+- `fpaste info` - Show server info (includes PoW difficulty)
+- `fpaste list` - List your pastes
+- `fpaste search --type image/*` - Search pastes by type/date
+- `fpaste update <id>` - Update paste content/metadata
+- `fpaste export -o dir/` - Export all pastes to directory
 - `fpaste pki status` - Show PKI status
 - `fpaste pki issue -n "name"` - Request client certificate
 - `fpaste pki revoke <serial>` - Revoke certificate
+- Automatic retry on PoW failure (max 5 attempts)
 - Config file for server URL and cert fingerprint
 - Downloadable via `curl https://server/client > fpaste`

@@ -144,6 +165,11 @@ These features will not be implemented:
 | 2024-12    | Minimal PKI                        | Self-contained mTLS without external CA
 | 2024-12    | Security tooling (ruff/bandit)     | Code quality and security scanning
 | 2024-12    | CI/CD with job dependencies        | Tests wait for lint to pass
+| 2024-12    | Anti-flood dynamic PoW             | Adaptive difficulty under attack
+| 2024-12    | IP-based rate limiting             | Per-IP request throttling
+| 2024-12    | Scheduled cleanup (in-process)     | No external cron needed
+| 2024-12    | CLI encrypt-by-default             | Security-first design
+| 2024-12    | CLI retry on PoW failure           | Graceful handling of stale tokens

 ## Review Schedule

--- a/TASKLIST.md
+++ b/TASKLIST.md
@@ -16,7 +16,6 @@ Prioritized, actionable tasks. Each task is small and completable in one session

 | Status | Task
 |--------|--------------------------------------------------------------
-| ☐      | Add paste listing for authenticated users
 | ☐      | Add rate limit headers (X-RateLimit-*)

 ## Priority 3: Quality
@@ -38,6 +37,12 @@ Prioritized, actionable tasks. Each task is small and completable in one session

 | Date       | Task
 |------------|--------------------------------------------------------------
+| 2024-12    | Implement anti-flood (dynamic PoW difficulty)
+| 2024-12    | Implement IP-based rate limiting
+| 2024-12    | Add scheduled cleanup (pastes, hashes, rate limits)
+| 2024-12    | Add CLI list/search/update/export commands
+| 2024-12    | Add CLI PoW retry (max 5 attempts)
+| 2024-12    | Add paste listing for authenticated users
 | 2024-12    | Implement minimal PKI (CA, issuance, revocation)
 | 2024-12    | Add security tooling (ruff, bandit, mypy, pip-audit)
 | 2024-12    | Create Makefile with dev workflow targets
--- a/TODO.md
+++ b/TODO.md
@@ -26,10 +26,13 @@ Unstructured intake buffer for ideas, issues, and observations. Items here are r

 ## Questions

- Should expired paste cleanup run in-process or via external cron?
 - Certificate renewal: reissue with same CN or require new request?
 - Should revoked certs be purged after grace period?

+## Resolved
+
+- Expired paste cleanup runs in-process via before_request hook (no cron needed)
+
 ## Debt

 - Mypy has pre-existing type errors (runs with --ignore-missing-imports)
--- a/documentation/api.md
+++ b/documentation/api.md
@@ -52,13 +52,30 @@ Host: localhost:5000
  "enabled": true,
  "nonce": "a1b2c3d4e5f6a7b8a1b2c3d4e5f6a7b8",
  "difficulty": 20,
+  "base_difficulty": 20,
+  "elevated": false,
  "expires": 1700000300,
  "token": "a1b2c3d4...:1700000300:20:signature"
 }
 ```

+**Response (PoW enabled, anti-flood active):**
+```json
+{
+  "enabled": true,
+  "nonce": "a1b2c3d4e5f6a7b8a1b2c3d4e5f6a7b8",
+  "difficulty": 24,
+  "base_difficulty": 20,
+  "elevated": true,
+  "expires": 1700000300,
+  "token": "a1b2c3d4...:1700000300:24:signature"
+}
+```
+
 The client must find a number `N` such that `SHA256(nonce + ":" + N)` has at least `difficulty` leading zero bits.

+When `elevated` is true, the difficulty has been increased due to high request volume (anti-flood protection). The `base_difficulty` shows the normal difficulty level.
+
 ---

 ### GET /health
@@ -123,6 +140,55 @@ Host: localhost:5000

 ---

+### GET /pastes
+
+List pastes for the authenticated user. Requires authentication.
+
+**Request:**
+```http
+GET /pastes HTTP/1.1
+Host: localhost:5000
+X-SSL-Client-SHA1: a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2
+```
+
+**Query Parameters:**
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `limit` | int | Maximum results (default: 100, max: 1000) |
+| `offset` | int | Skip first N results (default: 0) |
+| `type` | string | Filter by MIME type (glob pattern, e.g., `image/*`) |
+| `after` | int | Created after timestamp (Unix epoch) |
+| `before` | int | Created before timestamp (Unix epoch) |
+
+**Response (200 OK):**
+```json
+{
+  "pastes": [
+    {
+      "id": "a1b2c3d4e5f6",
+      "size": 1234,
+      "mime_type": "text/plain",
+      "created_at": "2024-12-20T10:30:00Z",
+      "expires_at": "2024-12-25T10:30:00Z",
+      "burn_after_read": false,
+      "password_protected": false
+    }
+  ],
+  "total": 42,
+  "limit": 100,
+  "offset": 0
+}
+```
+
+**Response (401 Unauthorized):**
+```json
+{
+  "error": "Authentication required"
+}
+```
+
+---
+
 ### POST /

 Create a new paste.
@@ -223,6 +289,7 @@ Password protected content
 - Minimum: disabled by default (`FLASKPASTE_MIN_SIZE`, e.g. 64 bytes for encryption enforcement)
 - Anonymous: 3 MiB (configurable via `FLASKPASTE_MAX_ANON`)
 - Authenticated: 50 MiB (configurable via `FLASKPASTE_MAX_AUTH`)
+
 ---

 ### GET /{id}