forked from claw/flaskpaste
docs: update for v1.4.0 features
- Add anti-flood, rate limiting, scheduled cleanup to feature lists - Update version to 1.4.0, test count to 205 - Document /pastes endpoint with query parameters - Add anti-flood fields to /challenge response - Update CLI docs with new commands (list, search, export) - Add decision log entries for recent features
This commit is contained in:
@@ -119,7 +119,7 @@ A self-hosted pastebin API that:
|
|||||||
|
|
||||||
## Current Status
|
## Current Status
|
||||||
|
|
||||||
**Version:** 1.2.0
|
**Version:** 1.4.0
|
||||||
|
|
||||||
```
|
```
|
||||||
┌─────────────────────────────────┬────────────────────────────────────────────┐
|
┌─────────────────────────────────┬────────────────────────────────────────────┐
|
||||||
@@ -134,6 +134,8 @@ A self-hosted pastebin API that:
|
|||||||
│ Paste expiry │ Complete
|
│ Paste expiry │ Complete
|
||||||
│ Content-hash deduplication │ Complete
|
│ Content-hash deduplication │ Complete
|
||||||
│ Proof-of-work │ Complete
|
│ Proof-of-work │ Complete
|
||||||
|
│ Anti-flood (dynamic PoW) │ Complete
|
||||||
|
│ IP-based rate limiting │ Complete
|
||||||
│ URL prefix support │ Complete
|
│ URL prefix support │ Complete
|
||||||
│ /client endpoint │ Complete
|
│ /client endpoint │ Complete
|
||||||
│ E2E encryption (CLI) │ Complete
|
│ E2E encryption (CLI) │ Complete
|
||||||
@@ -145,6 +147,8 @@ A self-hosted pastebin API that:
|
|||||||
│ Container deployment │ Complete
|
│ Container deployment │ Complete
|
||||||
│ Security tooling │ Complete
|
│ Security tooling │ Complete
|
||||||
│ CI/CD pipeline │ Complete
|
│ CI/CD pipeline │ Complete
|
||||||
│ Test suite │ 147 tests passing
|
│ Scheduled cleanup │ Complete
|
||||||
|
│ CLI paste listing/search │ Complete
|
||||||
|
│ Test suite │ 205 tests passing
|
||||||
└─────────────────────────────────┴────────────────────────────────────────────┘
|
└─────────────────────────────────┴────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|||||||
12
README.md
12
README.md
@@ -13,6 +13,8 @@ A lightweight, secure pastebin REST API built with Flask.
|
|||||||
- **Abuse prevention** - Content-hash deduplication throttles repeated identical submissions
|
- **Abuse prevention** - Content-hash deduplication throttles repeated identical submissions
|
||||||
- **Entropy enforcement** - Optional minimum entropy requirement to enforce client-side encryption
|
- **Entropy enforcement** - Optional minimum entropy requirement to enforce client-side encryption
|
||||||
- **Proof-of-work** - Configurable computational puzzle prevents automated spam
|
- **Proof-of-work** - Configurable computational puzzle prevents automated spam
|
||||||
|
- **Anti-flood** - Dynamic PoW difficulty increases under attack, decays when abuse stops
|
||||||
|
- **Rate limiting** - Per-IP request throttling with auth user multiplier
|
||||||
- **E2E encryption** - Client-side AES-256-GCM encryption with key in URL fragment (zero-knowledge)
|
- **E2E encryption** - Client-side AES-256-GCM encryption with key in URL fragment (zero-knowledge)
|
||||||
- **Burn-after-read** - Single-access pastes that auto-delete after first retrieval
|
- **Burn-after-read** - Single-access pastes that auto-delete after first retrieval
|
||||||
- **Custom expiry** - Per-paste expiry override via X-Expiry header
|
- **Custom expiry** - Per-paste expiry override via X-Expiry header
|
||||||
@@ -229,6 +231,16 @@ Configuration via environment variables:
|
|||||||
| `FLASKPASTE_POW_DIFFICULTY` | `20` | PoW difficulty (leading zero bits, 0=disabled) |
|
| `FLASKPASTE_POW_DIFFICULTY` | `20` | PoW difficulty (leading zero bits, 0=disabled) |
|
||||||
| `FLASKPASTE_POW_TTL` | `300` (5 min) | PoW challenge validity period |
|
| `FLASKPASTE_POW_TTL` | `300` (5 min) | PoW challenge validity period |
|
||||||
| `FLASKPASTE_POW_SECRET` | (auto) | Secret for signing PoW challenges |
|
| `FLASKPASTE_POW_SECRET` | (auto) | Secret for signing PoW challenges |
|
||||||
|
| `FLASKPASTE_ANTIFLOOD` | `1` | Enable anti-flood (dynamic PoW difficulty) |
|
||||||
|
| `FLASKPASTE_ANTIFLOOD_WINDOW` | `60` | Anti-flood measurement window (seconds) |
|
||||||
|
| `FLASKPASTE_ANTIFLOOD_THRESHOLD` | `5` | Requests per window before difficulty increase |
|
||||||
|
| `FLASKPASTE_ANTIFLOOD_STEP` | `2` | Difficulty bits added per threshold breach |
|
||||||
|
| `FLASKPASTE_ANTIFLOOD_MAX` | `28` | Maximum PoW difficulty |
|
||||||
|
| `FLASKPASTE_ANTIFLOOD_DECAY` | `60` | Seconds before difficulty decreases |
|
||||||
|
| `FLASKPASTE_RATE_LIMIT` | `1` | Enable IP-based rate limiting |
|
||||||
|
| `FLASKPASTE_RATE_WINDOW` | `60` | Rate limit window (seconds) |
|
||||||
|
| `FLASKPASTE_RATE_MAX` | `10` | Max requests per window (anon) |
|
||||||
|
| `FLASKPASTE_RATE_AUTH_MULT` | `5` | Multiplier for authenticated users |
|
||||||
| `FLASKPASTE_URL_PREFIX` | (empty) | URL prefix for reverse proxy deployments |
|
| `FLASKPASTE_URL_PREFIX` | (empty) | URL prefix for reverse proxy deployments |
|
||||||
| `FLASKPASTE_MIN_ENTROPY` | `0` | Min entropy bits/byte (0=disabled, 6.0=require encryption) |
|
| `FLASKPASTE_MIN_ENTROPY` | `0` | Min entropy bits/byte (0=disabled, 6.0=require encryption) |
|
||||||
| `FLASKPASTE_MIN_ENTROPY_SIZE` | `256` | Only check entropy for content >= this size |
|
| `FLASKPASTE_MIN_ENTROPY_SIZE` | `256` | Only check entropy for content >= this size |
|
||||||
|
|||||||
42
ROADMAP.md
42
ROADMAP.md
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
## Current State
|
## Current State
|
||||||
|
|
||||||
FlaskPaste v1.2.0 is deployed with PKI integration and comprehensive security tooling.
|
FlaskPaste v1.4.0 is deployed with comprehensive security hardening and abuse prevention.
|
||||||
|
|
||||||
**Implemented:**
|
**Implemented:**
|
||||||
- Full REST API (CRUD operations)
|
- Full REST API (CRUD operations)
|
||||||
@@ -11,6 +11,8 @@ FlaskPaste v1.2.0 is deployed with PKI integration and comprehensive security to
|
|||||||
- Minimal PKI (CA generation, certificate issuance, revocation)
|
- Minimal PKI (CA generation, certificate issuance, revocation)
|
||||||
- Content-hash deduplication (abuse prevention)
|
- Content-hash deduplication (abuse prevention)
|
||||||
- Proof-of-work spam prevention
|
- Proof-of-work spam prevention
|
||||||
|
- Anti-flood system (dynamic PoW difficulty under load)
|
||||||
|
- IP-based rate limiting (configurable per-IP limits)
|
||||||
- Entropy enforcement (require encrypted uploads)
|
- Entropy enforcement (require encrypted uploads)
|
||||||
- E2E encryption in CLI (AES-256-GCM, key in URL fragment)
|
- E2E encryption in CLI (AES-256-GCM, key in URL fragment)
|
||||||
- URL prefix support for reverse proxy deployments
|
- URL prefix support for reverse proxy deployments
|
||||||
@@ -18,11 +20,13 @@ FlaskPaste v1.2.0 is deployed with PKI integration and comprehensive security to
|
|||||||
- Automatic paste expiry
|
- Automatic paste expiry
|
||||||
- Burn-after-read pastes
|
- Burn-after-read pastes
|
||||||
- Custom expiry per paste
|
- Custom expiry per paste
|
||||||
|
- Scheduled cleanup (pastes, hashes, rate limits)
|
||||||
- Security headers and request tracing
|
- Security headers and request tracing
|
||||||
- Container deployment support
|
- Container deployment support
|
||||||
- Security tooling (ruff, bandit, mypy, pip-audit)
|
- Security tooling (ruff, bandit, mypy, pip-audit)
|
||||||
- CI/CD pipeline with lint, security, and test jobs
|
- CI/CD pipeline with lint, security, and test jobs
|
||||||
- Comprehensive test suite (147 tests)
|
- CLI with list, search, update, export commands
|
||||||
|
- Comprehensive test suite (205 tests)
|
||||||
|
|
||||||
## Phase 1: Hardening (Complete)
|
## Phase 1: Hardening (Complete)
|
||||||
|
|
||||||
@@ -38,7 +42,7 @@ Focus: Production readiness and operational excellence.
|
|||||||
│ 4 │ Proxy trust validation │ Done
|
│ 4 │ Proxy trust validation │ Done
|
||||||
│ 5 │ Proof-of-work spam prevention │ Done
|
│ 5 │ Proof-of-work spam prevention │ Done
|
||||||
│ 6 │ Entropy enforcement │ Done
|
│ 6 │ Entropy enforcement │ Done
|
||||||
│ 7 │ Test coverage > 90% │ Done (147 tests)
|
│ 7 │ Test coverage > 90% │ Done (205 tests)
|
||||||
│ 8 │ Documentation complete │ Done
|
│ 8 │ Documentation complete │ Done
|
||||||
└───┴─────────────────────────────────┴────────────────────────────────────┘
|
└───┴─────────────────────────────────┴────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
@@ -67,14 +71,26 @@ Focus: User-requested enhancements within scope.
|
|||||||
┌───┬─────────────────────────────────┬────────────────────────────────────┐
|
┌───┬─────────────────────────────────┬────────────────────────────────────┐
|
||||||
│ # │ Feature │ Status
|
│ # │ Feature │ Status
|
||||||
├───┼─────────────────────────────────┼────────────────────────────────────┤
|
├───┼─────────────────────────────────┼────────────────────────────────────┤
|
||||||
│ 1 │ E2E encryption (client-side) │ Done (CLI -e flag, zero-knowledge)
|
│ 1 │ E2E encryption (client-side) │ Done (CLI encrypts by default)
|
||||||
│ 2 │ URL prefix support │ Done
|
│ 2 │ URL prefix support │ Done
|
||||||
│ 3 │ Custom expiry per paste │ Done (X-Expiry header)
|
│ 3 │ Custom expiry per paste │ Done (X-Expiry header)
|
||||||
│ 4 │ Burn-after-read option │ Done (X-Burn-After-Read header)
|
│ 4 │ Burn-after-read option │ Done (X-Burn-After-Read header)
|
||||||
│ 5 │ Minimal PKI (CA + issuance) │ Done
|
│ 5 │ Minimal PKI (CA + issuance) │ Done
|
||||||
|
│ 6 │ Anti-flood (dynamic PoW) │ Done (v1.4.0)
|
||||||
|
│ 7 │ IP-based rate limiting │ Done (v1.4.0)
|
||||||
|
│ 8 │ Scheduled cleanup │ Done (v1.4.0)
|
||||||
└───┴─────────────────────────────────┴────────────────────────────────────┘
|
└───┴─────────────────────────────────┴────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Anti-Flood System (v1.4.0)
|
||||||
|
|
||||||
|
Dynamic proof-of-work difficulty that increases under abuse:
|
||||||
|
- Base difficulty: 20 bits (configurable)
|
||||||
|
- Threshold: 5 requests per 60s window triggers increase
|
||||||
|
- Step: +2 bits per threshold breach
|
||||||
|
- Maximum: 28 bits
|
||||||
|
- Decay: Returns to base after 60s of normal traffic
|
||||||
|
|
||||||
### PKI Features
|
### PKI Features
|
||||||
|
|
||||||
Integrated certificate authority for mTLS:
|
Integrated certificate authority for mTLS:
|
||||||
@@ -103,15 +119,20 @@ Focus: Integration with external systems.
|
|||||||
|
|
||||||
### CLI Client (Complete)
|
### CLI Client (Complete)
|
||||||
|
|
||||||
Standalone Python CLI with encryption and PKI support:
|
Standalone Python CLI with encryption, PKI, and paste management:
|
||||||
- `fpaste create file.txt` - Create paste from file
|
- `fpaste file.txt` - Create encrypted paste (file path shortcut)
|
||||||
- `fpaste create -e file.txt` - Create encrypted paste (E2E)
|
- `fpaste create -E file.txt` - Create unencrypted paste
|
||||||
- `fpaste get <id>` - Get paste (auto-decrypts with URL fragment key)
|
- `fpaste get <id>` - Get paste (auto-decrypts with URL fragment key)
|
||||||
- `fpaste delete <id>` - Delete paste
|
- `fpaste delete <id>` - Delete paste
|
||||||
- `fpaste info` - Show server info
|
- `fpaste info` - Show server info (includes PoW difficulty)
|
||||||
|
- `fpaste list` - List your pastes
|
||||||
|
- `fpaste search --type image/*` - Search pastes by type/date
|
||||||
|
- `fpaste update <id>` - Update paste content/metadata
|
||||||
|
- `fpaste export -o dir/` - Export all pastes to directory
|
||||||
- `fpaste pki status` - Show PKI status
|
- `fpaste pki status` - Show PKI status
|
||||||
- `fpaste pki issue -n "name"` - Request client certificate
|
- `fpaste pki issue -n "name"` - Request client certificate
|
||||||
- `fpaste pki revoke <serial>` - Revoke certificate
|
- `fpaste pki revoke <serial>` - Revoke certificate
|
||||||
|
- Automatic retry on PoW failure (max 5 attempts)
|
||||||
- Config file for server URL and cert fingerprint
|
- Config file for server URL and cert fingerprint
|
||||||
- Downloadable via `curl https://server/client > fpaste`
|
- Downloadable via `curl https://server/client > fpaste`
|
||||||
|
|
||||||
@@ -144,6 +165,11 @@ These features will not be implemented:
|
|||||||
| 2024-12 | Minimal PKI | Self-contained mTLS without external CA
|
| 2024-12 | Minimal PKI | Self-contained mTLS without external CA
|
||||||
| 2024-12 | Security tooling (ruff/bandit) | Code quality and security scanning
|
| 2024-12 | Security tooling (ruff/bandit) | Code quality and security scanning
|
||||||
| 2024-12 | CI/CD with job dependencies | Tests wait for lint to pass
|
| 2024-12 | CI/CD with job dependencies | Tests wait for lint to pass
|
||||||
|
| 2024-12 | Anti-flood dynamic PoW | Adaptive difficulty under attack
|
||||||
|
| 2024-12 | IP-based rate limiting | Per-IP request throttling
|
||||||
|
| 2024-12 | Scheduled cleanup (in-process) | No external cron needed
|
||||||
|
| 2024-12 | CLI encrypt-by-default | Security-first design
|
||||||
|
| 2024-12 | CLI retry on PoW failure | Graceful handling of stale tokens
|
||||||
|
|
||||||
## Review Schedule
|
## Review Schedule
|
||||||
|
|
||||||
|
|||||||
@@ -16,7 +16,6 @@ Prioritized, actionable tasks. Each task is small and completable in one session
|
|||||||
|
|
||||||
| Status | Task
|
| Status | Task
|
||||||
|--------|--------------------------------------------------------------
|
|--------|--------------------------------------------------------------
|
||||||
| ☐ | Add paste listing for authenticated users
|
|
||||||
| ☐ | Add rate limit headers (X-RateLimit-*)
|
| ☐ | Add rate limit headers (X-RateLimit-*)
|
||||||
|
|
||||||
## Priority 3: Quality
|
## Priority 3: Quality
|
||||||
@@ -38,6 +37,12 @@ Prioritized, actionable tasks. Each task is small and completable in one session
|
|||||||
|
|
||||||
| Date | Task
|
| Date | Task
|
||||||
|------------|--------------------------------------------------------------
|
|------------|--------------------------------------------------------------
|
||||||
|
| 2024-12 | Implement anti-flood (dynamic PoW difficulty)
|
||||||
|
| 2024-12 | Implement IP-based rate limiting
|
||||||
|
| 2024-12 | Add scheduled cleanup (pastes, hashes, rate limits)
|
||||||
|
| 2024-12 | Add CLI list/search/update/export commands
|
||||||
|
| 2024-12 | Add CLI PoW retry (max 5 attempts)
|
||||||
|
| 2024-12 | Add paste listing for authenticated users
|
||||||
| 2024-12 | Implement minimal PKI (CA, issuance, revocation)
|
| 2024-12 | Implement minimal PKI (CA, issuance, revocation)
|
||||||
| 2024-12 | Add security tooling (ruff, bandit, mypy, pip-audit)
|
| 2024-12 | Add security tooling (ruff, bandit, mypy, pip-audit)
|
||||||
| 2024-12 | Create Makefile with dev workflow targets
|
| 2024-12 | Create Makefile with dev workflow targets
|
||||||
|
|||||||
5
TODO.md
5
TODO.md
@@ -26,10 +26,13 @@ Unstructured intake buffer for ideas, issues, and observations. Items here are r
|
|||||||
|
|
||||||
## Questions
|
## Questions
|
||||||
|
|
||||||
- Should expired paste cleanup run in-process or via external cron?
|
|
||||||
- Certificate renewal: reissue with same CN or require new request?
|
- Certificate renewal: reissue with same CN or require new request?
|
||||||
- Should revoked certs be purged after grace period?
|
- Should revoked certs be purged after grace period?
|
||||||
|
|
||||||
|
## Resolved
|
||||||
|
|
||||||
|
- Expired paste cleanup runs in-process via before_request hook (no cron needed)
|
||||||
|
|
||||||
## Debt
|
## Debt
|
||||||
|
|
||||||
- Mypy has pre-existing type errors (runs with --ignore-missing-imports)
|
- Mypy has pre-existing type errors (runs with --ignore-missing-imports)
|
||||||
|
|||||||
@@ -52,13 +52,30 @@ Host: localhost:5000
|
|||||||
"enabled": true,
|
"enabled": true,
|
||||||
"nonce": "a1b2c3d4e5f6a7b8a1b2c3d4e5f6a7b8",
|
"nonce": "a1b2c3d4e5f6a7b8a1b2c3d4e5f6a7b8",
|
||||||
"difficulty": 20,
|
"difficulty": 20,
|
||||||
|
"base_difficulty": 20,
|
||||||
|
"elevated": false,
|
||||||
"expires": 1700000300,
|
"expires": 1700000300,
|
||||||
"token": "a1b2c3d4...:1700000300:20:signature"
|
"token": "a1b2c3d4...:1700000300:20:signature"
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Response (PoW enabled, anti-flood active):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"enabled": true,
|
||||||
|
"nonce": "a1b2c3d4e5f6a7b8a1b2c3d4e5f6a7b8",
|
||||||
|
"difficulty": 24,
|
||||||
|
"base_difficulty": 20,
|
||||||
|
"elevated": true,
|
||||||
|
"expires": 1700000300,
|
||||||
|
"token": "a1b2c3d4...:1700000300:24:signature"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
The client must find a number `N` such that `SHA256(nonce + ":" + N)` has at least `difficulty` leading zero bits.
|
The client must find a number `N` such that `SHA256(nonce + ":" + N)` has at least `difficulty` leading zero bits.
|
||||||
|
|
||||||
|
When `elevated` is true, the difficulty has been increased due to high request volume (anti-flood protection). The `base_difficulty` shows the normal difficulty level.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### GET /health
|
### GET /health
|
||||||
@@ -123,6 +140,55 @@ Host: localhost:5000
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
### GET /pastes
|
||||||
|
|
||||||
|
List pastes for the authenticated user. Requires authentication.
|
||||||
|
|
||||||
|
**Request:**
|
||||||
|
```http
|
||||||
|
GET /pastes HTTP/1.1
|
||||||
|
Host: localhost:5000
|
||||||
|
X-SSL-Client-SHA1: a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2
|
||||||
|
```
|
||||||
|
|
||||||
|
**Query Parameters:**
|
||||||
|
| Parameter | Type | Description |
|
||||||
|
|-----------|------|-------------|
|
||||||
|
| `limit` | int | Maximum results (default: 100, max: 1000) |
|
||||||
|
| `offset` | int | Skip first N results (default: 0) |
|
||||||
|
| `type` | string | Filter by MIME type (glob pattern, e.g., `image/*`) |
|
||||||
|
| `after` | int | Created after timestamp (Unix epoch) |
|
||||||
|
| `before` | int | Created before timestamp (Unix epoch) |
|
||||||
|
|
||||||
|
**Response (200 OK):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"pastes": [
|
||||||
|
{
|
||||||
|
"id": "a1b2c3d4e5f6",
|
||||||
|
"size": 1234,
|
||||||
|
"mime_type": "text/plain",
|
||||||
|
"created_at": "2024-12-20T10:30:00Z",
|
||||||
|
"expires_at": "2024-12-25T10:30:00Z",
|
||||||
|
"burn_after_read": false,
|
||||||
|
"password_protected": false
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"total": 42,
|
||||||
|
"limit": 100,
|
||||||
|
"offset": 0
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response (401 Unauthorized):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"error": "Authentication required"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
### POST /
|
### POST /
|
||||||
|
|
||||||
Create a new paste.
|
Create a new paste.
|
||||||
@@ -223,6 +289,7 @@ Password protected content
|
|||||||
- Minimum: disabled by default (`FLASKPASTE_MIN_SIZE`, e.g. 64 bytes for encryption enforcement)
|
- Minimum: disabled by default (`FLASKPASTE_MIN_SIZE`, e.g. 64 bytes for encryption enforcement)
|
||||||
- Anonymous: 3 MiB (configurable via `FLASKPASTE_MAX_ANON`)
|
- Anonymous: 3 MiB (configurable via `FLASKPASTE_MAX_ANON`)
|
||||||
- Authenticated: 50 MiB (configurable via `FLASKPASTE_MAX_AUTH`)
|
- Authenticated: 50 MiB (configurable via `FLASKPASTE_MAX_AUTH`)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### GET /{id}
|
### GET /{id}
|
||||||
|
|||||||
Reference in New Issue
Block a user