claw/flaskpaste
1
0
Fork 1
forked from username/flaskpaste
Code Pull Requests Activity

update project documentation

Browse Source
This commit is contained in:
Username
2025-12-20 17:20:40 +01:00
parent 713052de3d
commit 260d6e894d
5 changed files with 197 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
PROJECT.md
View File

@@ -58,6 +58,12 @@ A self-hosted pastebin API that:
- Configurable size limits (anon vs authenticated)
- Time-based expiry with access-touch semantics
- Content-hash deduplication for abuse prevention
- Proof-of-work spam prevention
- Entropy enforcement for encrypted content
- Client-side E2E encryption (CLI)
- Burn-after-read pastes
- Custom expiry per paste
- URL prefix for reverse proxy deployments
- Security headers (HSTS, CSP, X-Frame-Options, etc.)
- Request tracing and structured logging
- Container deployment support
@@ -102,3 +108,32 @@ A self-hosted pastebin API that:
│ Python │ 3.11+
└─────────────────┴──────────────────────────────────────────────────────────┘
```
## Current Status
**Version:** 1.1.0
```
┌─────────────────────────────────┬────────────────────────────────────────────┐
│ Feature │ Status
├─────────────────────────────────┼────────────────────────────────────────────┤
│ Core API (CRUD) │ Complete
│ Binary content support │ Complete
│ MIME detection │ Complete
│ Client cert authentication │ Complete
│ Size limits │ Complete
│ Paste expiry │ Complete
│ Content-hash deduplication │ Complete
│ Proof-of-work │ Complete
│ URL prefix support │ Complete
│ /client endpoint │ Complete
│ E2E encryption (CLI) │ Complete
│ Entropy enforcement │ Complete
│ Burn-after-read │ Complete
│ Custom expiry │ Complete
│ Security headers │ Complete
│ Request tracing │ Complete
│ Container deployment │ Complete
│ Test suite │ 113 tests passing
└─────────────────────────────────┴────────────────────────────────────────────┘
```

96
README.md
View File

@@ -11,7 +11,12 @@ A lightweight, secure pastebin REST API built with Flask.
- **Automatic expiry** - Pastes expire after configurable period of inactivity
- **Size limits** - Configurable limits for anonymous and authenticated users
- **Abuse prevention** - Content-hash deduplication throttles repeated identical submissions
- **Entropy enforcement** - Optional minimum entropy requirement to enforce client-side encryption
- **Proof-of-work** - Configurable computational puzzle prevents automated spam
- **E2E encryption** - Client-side AES-256-GCM encryption with key in URL fragment (zero-knowledge)
- **Burn-after-read** - Single-access pastes that auto-delete after first retrieval
- **Custom expiry** - Per-paste expiry override via X-Expiry header
- **Password protection** - Optional paste passwords with PBKDF2 hashing
- **Security headers** - HSTS, CSP, X-Frame-Options, Cache-Control, and more
- **CLI client** - Standalone `fpaste` command-line tool included
- **Request tracing** - X-Request-ID support for log correlation
@@ -39,6 +44,7 @@ python run.py
| `GET /` | API information and usage |
| `GET /health` | Health check (returns DB status) |
| `GET /challenge` | Get proof-of-work challenge |
| `GET /client` | Download fpaste CLI client |
| `POST /` | Create a new paste |
| `GET /<id>` | Retrieve paste metadata |
| `HEAD /<id>` | Retrieve paste metadata (headers only) |
@@ -84,7 +90,17 @@ curl -X DELETE \
## CLI Client
A standalone command-line client `fpaste` is included (no external dependencies).
A standalone command-line client `fpaste` is included. For E2E encryption, install the optional `cryptography` package.
### Installation
```bash
# Download from running server
curl -sS https://paste.example.com/client > fpaste && chmod +x fpaste
# Optional: enable encryption support
pip install cryptography
```
### Basic Usage
@@ -95,9 +111,25 @@ A standalone command-line client `fpaste` is included (no external dependencies)
# Create paste from stdin
echo "Hello" | ./fpaste
# Create encrypted paste (E2E, zero-knowledge)
./fpaste create -e secret.txt
# Returns: https://paste.example.com/abc123#<key>
# Create burn-after-read paste (single access, auto-deletes)
./fpaste create -b secret.txt
# Create paste with custom expiry (1 hour)
./fpaste create -x 3600 temp.txt
# Combine options: encrypted + burn-after-read
./fpaste create -e -b secret.txt
# Get paste content
./fpaste get abc12345
# Get encrypted paste (auto-decrypts if URL has #key fragment)
./fpaste get "https://paste.example.com/abc123#<key>"
# Get paste metadata
./fpaste get -m abc12345
@@ -108,6 +140,17 @@ echo "Hello" | ./fpaste
./fpaste info
```
### End-to-End Encryption
The `-e` flag encrypts content client-side using AES-256-GCM before upload:
- Key is generated locally and never sent to server
- Key is appended to URL as fragment (`#...`) which browsers never transmit
- Server stores only opaque ciphertext
- Retrieval auto-detects `#key` fragment and decrypts locally
This provides true zero-knowledge storage: the server cannot read your content.
### Configuration
Set server URL and authentication via environment or config file:
@@ -122,6 +165,50 @@ server = https://paste.example.com
cert_sha1 = your-cert-fingerprint
```
### mTLS Client Certificate Authentication
The CLI supports mutual TLS (mTLS) for direct client certificate authentication:
```bash
# Environment variables
export FLASKPASTE_CLIENT_CERT="/path/to/client.crt"
export FLASKPASTE_CLIENT_KEY="/path/to/client.key"
export FLASKPASTE_CA_CERT="/path/to/ca.crt" # Optional CA verification
# Or config file (~/.config/fpaste/config)
client_cert = /path/to/client.crt
client_key = /path/to/client.key
ca_cert = /path/to/ca.crt
```
When client certificates are configured, fpaste performs mTLS authentication directly with the server (or TLS-terminating proxy). This is more secure than header-based authentication as the certificate is validated at the TLS layer.
### Generating Client Certificates
The CLI includes a built-in certificate generator for mTLS authentication:
```bash
# Generate EC certificate (recommended, fast key generation)
./fpaste cert
# Outputs fingerprint to stdout, details to stderr
# Generate and auto-configure fpaste
./fpaste cert --configure
# Creates ~/.config/fpaste/client.{crt,key} and updates config
# Custom options
./fpaste cert -a rsa -b 4096 -d 730 -n "my-client" --configure
# RSA 4096-bit key, 2-year validity, custom common name
# Supported curves for EC: secp256r1, secp384r1 (default), secp521r1
./fpaste cert -c secp256r1 --configure
```
The generated certificate includes:
- `CLIENT_AUTH` extended key usage for mTLS
- SHA1 fingerprint compatible with `X-SSL-Client-SHA1` header
- Self-signed with configurable validity period
## Configuration
Configuration via environment variables:
@@ -140,6 +227,10 @@ Configuration via environment variables:
| `FLASKPASTE_POW_DIFFICULTY` | `20` | PoW difficulty (leading zero bits, 0=disabled) |
| `FLASKPASTE_POW_TTL` | `300` (5 min) | PoW challenge validity period |
| `FLASKPASTE_POW_SECRET` | (auto) | Secret for signing PoW challenges |
| `FLASKPASTE_URL_PREFIX` | (empty) | URL prefix for reverse proxy deployments |
| `FLASKPASTE_MIN_ENTROPY` | `0` | Min entropy bits/byte (0=disabled, 6.0=require encryption) |
| `FLASKPASTE_MIN_ENTROPY_SIZE` | `256` | Only check entropy for content >= this size |
| `FLASKPASTE_MAX_EXPIRY` | `2592000` (30 days) | Maximum custom expiry allowed |
## Authentication
@@ -216,6 +307,9 @@ flaskpaste/
- **Ownership enforcement** - Only owners can delete their pastes
- **Size limits** - Prevents resource exhaustion attacks
- **Abuse prevention** - Content-hash deduplication prevents spam flooding
- **Entropy enforcement** - Optional minimum entropy rejects low-entropy (plaintext) uploads
- **E2E encryption** - Client-side encryption keeps server zero-knowledge
- **Burn-after-read** - Single-use pastes for sensitive data
- **Security headers** - HSTS, CSP, X-Frame-Options, X-Content-Type-Options, Cache-Control
- **Request tracing** - X-Request-ID for log correlation and debugging
- **Proxy trust** - Optional `X-Proxy-Secret` validation to prevent header spoofing

67
ROADMAP.md
View File

@@ -2,19 +2,24 @@
## Current State
FlaskPaste v1.0 is feature-complete for its core mission: a secure, minimal pastebin API.
FlaskPaste v1.1.0 is deployed and feature-complete for its core mission: a secure, minimal pastebin API with zero-knowledge encryption support.
**Implemented:**
- Full REST API (CRUD operations)
- Binary content support with magic-byte MIME detection
- Client certificate authentication
- Content-hash deduplication (abuse prevention)
- Proof-of-work spam prevention
- Entropy enforcement (require encrypted uploads)
- E2E encryption in CLI (AES-256-GCM, key in URL fragment)
- URL prefix support for reverse proxy deployments
- /client endpoint for CLI distribution
- Automatic paste expiry
- Security headers and request tracing
- Container deployment support
- Comprehensive test suite
- Comprehensive test suite (98 tests)
## Phase 1: Hardening (Current)
## Phase 1: Hardening (Complete)
Focus: Production readiness and operational excellence.
@@ -22,12 +27,14 @@ Focus: Production readiness and operational excellence.
┌───┬─────────────────────────────────────┬────────────────────────────────────┐
│ # │ Milestone │ Status
├───┼─────────────────────────────────────┼────────────────────────────────────┤
│ 1 │ Abuse prevention (dedup) │ Implemented (pending commit)
│ 1 │ Abuse prevention (dedup) │ Done
│ 2 │ Security headers complete │ Done
│ 3 │ Request tracing (X-Request-ID) │ Done
│ 4 │ Proxy trust validation │ Done
│ 5 │ Test coverage > 90% │ In progress
│ 6 │ Documentation complete │ In progress
│ 5 │ Proof-of-work spam prevention │ Done
│ 6 │ Entropy enforcement │ Done
│ 7 │ Test coverage > 90% │ Done (98 tests)
│ 8 │ Documentation complete │ Done
└───┴─────────────────────────────────────┴────────────────────────────────────┘
```
@@ -64,19 +71,19 @@ Replace text logs with JSON format:
- Consistent field names across all log entries
- Compatible with log aggregation (Loki, ELK)
## Phase 3: Features
## Phase 3: Features (Complete)
Focus: User-requested enhancements within scope.
```
┌───┬─────────────────────────────────────┬────────────────────────────────────┐
│ # │ Feature │ Complexity
│ # │ Feature │ Status
├───┼─────────────────────────────────────┼────────────────────────────────────┤
│ 1 │ Paste encryption (server-side) │ Medium
│ 2 │ Custom expiry per paste │ Low
│ 3 │ Paste size in response headers │ Low
│ 4 │ Burn-after-read option │ Low
│ 5 │ Paste password protection │ Medium
│ 1 │ E2E encryption (client-side) │ Done (CLI -e flag, zero-knowledge)
│ 2 │ URL prefix support │ Done
│ 3 │ Custom expiry per paste │ Done (X-Expiry header)
│ 4 │ Burn-after-read option │ Done (X-Burn-After-Read header)
│ 5 │ Paste password protection │ Planned
└───┴─────────────────────────────────────┴────────────────────────────────────┘
```
@@ -100,23 +107,26 @@ Focus: Integration with external systems.
```
┌───┬─────────────────────────────────────┬────────────────────────────────────┐
│ # │ Integration │ Purpose
│ # │ Integration │ Status
├───┼─────────────────────────────────────┼────────────────────────────────────┤
│ 1 │ CLI client (fpaste) │ User convenience
│ 2 │ Neovim/Vim plugin │ Editor integration
│ 3 │ Shell aliases/functions │ Workflow integration
│ 4 │ Webhook notifications │ Automation triggers
│ 1 │ CLI client (fpaste) │ Done (with E2E encryption)
│ 2 │ /client endpoint │ Done (downloadable CLI)
│ 3 │ Neovim/Vim plugin │ Planned
│ 4 │ Shell aliases/functions │ Planned
│ 5 │ Webhook notifications │ Planned
└───┴─────────────────────────────────────┴────────────────────────────────────┘
```
### CLI Client
### CLI Client (Complete)
Standalone Python CLI:
- `fpaste < file.txt` - Create paste from stdin
- `fpaste file.txt` - Create paste from file
- `fpaste -g <id>` - Get paste
- `fpaste -d <id>` - Delete paste
- Config file for server URL and cert path
Standalone Python CLI with encryption support:
- `fpaste create file.txt` - Create paste from file
- `fpaste create -e file.txt` - Create encrypted paste (E2E)
- `fpaste get <id>` - Get paste (auto-decrypts with URL fragment key)
- `fpaste delete <id>` - Delete paste
- `fpaste info` - Show server info
- Config file for server URL and cert fingerprint
- Downloadable via `curl https://server/client > fpaste`
## Non-Goals (Explicit)
@@ -137,6 +147,13 @@ These features will not be implemented:
| 2024-11 | No web UI | API-first; reduces attack surface
| 2024-11 | Client cert auth | Integrates with existing PKI
| 2024-12 | Content-hash dedup | Prevent spam without IP tracking
| 2024-12 | Proof-of-work | Computational cost deters spam bots
| 2024-12 | Client-side E2E encryption | Zero-knowledge; key in URL fragment
| 2024-12 | Entropy enforcement | Heuristic to require encrypted uploads
| 2024-12 | URL prefix support | Reverse proxy path-based routing
| 2024-12 | Burn-after-read | Single-use pastes for sensitive data
| 2024-12 | Custom expiry | Per-paste TTL override
| 2024-12 | Multi-stage Containerfile | Smaller production images
## Review Schedule

36
TASKLIST.md
View File

@@ -4,45 +4,47 @@ Prioritized, actionable tasks. Each task is small and completable in one session
---
## Priority 1: Pending Commit
## Priority 1: Operations
| Status | Task
|--------|--------------------------------------------------------------
| ☐ | Commit abuse prevention feature (dedup changes in routes.py, config.py, database.py)
| ☐ | Commit documentation updates (api.md, README.md)
| ☐ | Add /metrics endpoint skeleton (for future Prometheus)
| ☐ | Add structured logging option via environment variable
## Priority 2: Test Coverage
## Priority 2: Features
| Status | Task
|--------|--------------------------------------------------------------
| ☐ | Run test suite, verify all tests pass
| ☐ | Add test for dedup window expiry behavior
| ☐ | Add test for concurrent identical submissions
| ☐ | Add test for MIME detection edge cases (empty content, truncated headers)
| ☐ | Measure and document test coverage percentage
| ☐ | Add paste listing for authenticated users
| ☐ | Add paste password protection
## Priority 3: Documentation
| Status | Task
|--------|--------------------------------------------------------------
| ☐ | Add deployment examples to documentation/deployment.md
| ☐ | Document environment variables in one canonical location
| ☐ | Add troubleshooting section to README.md
| ☐ | Add deployment examples (Kubernetes, systemd)
| ☐ | Create CONTRIBUTING.md with development setup
## Priority 4: Operations
## Priority 4: Testing
| Status | Task
|--------|--------------------------------------------------------------
| ☑ | Add SQLite WAL mode for better concurrency
| ☐ | Add /metrics endpoint skeleton (for future Prometheus)
| ☐ | Add structured logging option via environment variable
| ☐ | Optimize container build with multi-stage Containerfile
| ☐ | Add test for concurrent identical submissions
| ☐ | Add integration tests for container deployment
## Completed
| Date | Task
|------------|--------------------------------------------------------------
| 2024-12 | Implement burn-after-read option
| 2024-12 | Implement custom expiry per paste
| 2024-12 | Optimize Containerfile with multi-stage build
| 2024-12 | Implement E2E encryption in CLI (AES-256-GCM)
| 2024-12 | Implement entropy enforcement
| 2024-12 | Add /client endpoint for CLI download
| 2024-12 | Add URL prefix support
| 2024-12 | Implement proof-of-work spam prevention
| 2024-12 | Update documentation for v1.1.0
| 2024-12 | Add HEAD method for paste endpoints
| 2024-12 | Add SQLite WAL mode for better concurrency
| 2024-12 | Implement content-hash deduplication

13
TODO.md
View File

@@ -10,20 +10,20 @@ Unstructured intake buffer for ideas, issues, and observations. Items here are r
- Structured JSON logging for log aggregation compatibility
- Burn-after-read paste option
- Custom expiry header for per-paste TTL
- CLI client tool (fpaste) for easier usage
- Rate limit headers in responses (X-RateLimit-*)
- Paste compression for large text content
- Optional paste encryption with user-provided key
- ETag support for conditional requests
- HEAD method support for metadata without body
- Paste listing for authenticated users (their own pastes only)
- Neovim/Vim plugin for editor integration
- Webhook notifications for paste events
## Observations
- Current abuse prevention uses content-hash; IP-based limiting delegated to proxy
- Abuse prevention uses content-hash dedup + PoW + entropy enforcement
- SQLite WAL mode could improve concurrent read performance
- Container image size could be reduced with multi-stage build
- Test coverage could include more edge cases for MIME detection
- E2E encryption in CLI uses cryptography package (optional dependency)
- Entropy check has size threshold to avoid false positives on small data
## Questions
@@ -34,10 +34,9 @@ Unstructured intake buffer for ideas, issues, and observations. Items here are r
## Debt
- Dedup feature changes pending commit
- Documentation could include more deployment examples
- No integration tests for container deployment
- Missing test for concurrent paste creation
- Could add more deployment examples (Kubernetes, systemd)
## External Dependencies