forked from claw/flaskpaste
flaskpaste: initial commit with security hardening
Features: - REST API for text/binary pastes with MIME detection - Client certificate auth via X-SSL-Client-SHA1 header - SQLite with WAL mode for concurrent access - Automatic paste expiry with LRU cleanup Security: - HSTS, CSP, X-Frame-Options, X-Content-Type-Options - Cache-Control: no-store for sensitive responses - X-Request-ID tracing for log correlation - X-Proxy-Secret validation for defense-in-depth - Parameterized queries, input validation - Size limits (3 MiB anon, 50 MiB auth) Includes /health endpoint, container support, and 70 tests.
This commit is contained in:
Username
333
documentation/api.md
Normal file
333
documentation/api.md
Normal file
@@ -0,0 +1,333 @@
|
||||
# FlaskPaste API Reference
|
||||
|
||||
## Overview
|
||||
|
||||
FlaskPaste provides a RESTful API for creating, retrieving, and deleting text and binary pastes.
|
||||
|
||||
**Base URL:** `http://your-server:5000/`
|
||||
|
||||
**Content Types:**
|
||||
- Requests: `application/json`, `text/plain`, `application/octet-stream`, or any binary type
|
||||
- Responses: `application/json` for metadata, original MIME type for raw content
|
||||
|
||||
## Authentication
|
||||
|
||||
Authentication is optional and uses client certificate fingerprints passed via the `X-SSL-Client-SHA1` header.
|
||||
|
||||
```http
|
||||
X-SSL-Client-SHA1: a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2
|
||||
```
|
||||
|
||||
The fingerprint must be exactly 40 lowercase hexadecimal characters (SHA1).
|
||||
|
||||
**Benefits of authentication:**
|
||||
- Larger upload limit (50 MiB vs 3 MiB)
|
||||
- Ability to delete owned pastes
|
||||
|
||||
---
|
||||
|
||||
## Endpoints
|
||||
|
||||
### GET /health
|
||||
|
||||
Health check endpoint for load balancers and monitoring.
|
||||
|
||||
**Request:**
|
||||
```http
|
||||
GET /health HTTP/1.1
|
||||
Host: localhost:5000
|
||||
```
|
||||
|
||||
**Response (200 OK):**
|
||||
```json
|
||||
{
|
||||
"status": "healthy",
|
||||
"database": "ok"
|
||||
}
|
||||
```
|
||||
|
||||
**Response (503 Service Unavailable):**
|
||||
```json
|
||||
{
|
||||
"status": "unhealthy",
|
||||
"database": "error"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### GET /
|
||||
|
||||
Returns API information and usage instructions.
|
||||
|
||||
**Request:**
|
||||
```http
|
||||
GET / HTTP/1.1
|
||||
Host: localhost:5000
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"name": "FlaskPaste",
|
||||
"version": "1.0.0",
|
||||
"endpoints": {
|
||||
"GET /": "API information",
|
||||
"GET /health": "Health check",
|
||||
"POST /": "Create paste",
|
||||
"GET /<id>": "Retrieve paste metadata",
|
||||
"GET /<id>/raw": "Retrieve raw paste content",
|
||||
"DELETE /<id>": "Delete paste"
|
||||
},
|
||||
"usage": {
|
||||
"raw": "curl --data-binary @file.txt http://host/",
|
||||
"pipe": "cat file.txt | curl --data-binary @- http://host/",
|
||||
"json": "curl -H 'Content-Type: application/json' -d '{\"content\":\"...\"}' http://host/"
|
||||
},
|
||||
"note": "Use --data-binary (not -d) to preserve newlines"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### POST /
|
||||
|
||||
Create a new paste.
|
||||
|
||||
**Request (Raw Binary):**
|
||||
```http
|
||||
POST / HTTP/1.1
|
||||
Host: localhost:5000
|
||||
Content-Type: application/octet-stream
|
||||
|
||||
```
|
||||
|
||||
**Request (JSON):**
|
||||
```http
|
||||
POST / HTTP/1.1
|
||||
Host: localhost:5000
|
||||
Content-Type: application/json
|
||||
|
||||
```
|
||||
|
||||
**Response (201 Created):**
|
||||
```json
|
||||
{
|
||||
"id": "abc12345",
|
||||
"url": "/abc12345",
|
||||
"raw": "/abc12345/raw",
|
||||
"mime_type": "text/plain",
|
||||
"created_at": 1700000000,
|
||||
"owner": "a1b2c3..." // Only present if authenticated
|
||||
}
|
||||
```
|
||||
|
||||
**Errors:**
|
||||
| Code | Description |
|
||||
|------|-------------|
|
||||
| 400 | No content provided |
|
||||
| 413 | Paste too large |
|
||||
|
||||
**Size Limits:**
|
||||
- Anonymous: 3 MiB (configurable via `FLASKPASTE_MAX_ANON`)
|
||||
- Authenticated: 50 MiB (configurable via `FLASKPASTE_MAX_AUTH`)
|
||||
|
||||
---
|
||||
|
||||
### GET /{id}
|
||||
|
||||
Retrieve paste metadata.
|
||||
|
||||
**Request:**
|
||||
```http
|
||||
GET /abc12345 HTTP/1.1
|
||||
Host: localhost:5000
|
||||
```
|
||||
|
||||
**Response (200 OK):**
|
||||
```json
|
||||
{
|
||||
"id": "abc12345",
|
||||
"mime_type": "text/plain",
|
||||
"size": 1234,
|
||||
"created_at": 1700000000,
|
||||
"raw": "/abc12345/raw"
|
||||
}
|
||||
```
|
||||
|
||||
**Errors:**
|
||||
| Code | Description |
|
||||
|------|-------------|
|
||||
| 400 | Invalid paste ID format |
|
||||
| 404 | Paste not found |
|
||||
|
||||
---
|
||||
|
||||
### GET /{id}/raw
|
||||
|
||||
Retrieve raw paste content with correct MIME type.
|
||||
|
||||
**Request:**
|
||||
```http
|
||||
GET /abc12345/raw HTTP/1.1
|
||||
Host: localhost:5000
|
||||
```
|
||||
|
||||
**Response (200 OK):**
|
||||
```http
|
||||
HTTP/1.1 200 OK
|
||||
Content-Type: image/png
|
||||
Content-Disposition: inline
|
||||
|
||||
<binary content>
|
||||
```
|
||||
|
||||
- `Content-Disposition: inline` is set for `image/*` and `text/*` types
|
||||
- Content-Type matches the detected/stored MIME type
|
||||
|
||||
**Errors:**
|
||||
| Code | Description |
|
||||
|------|-------------|
|
||||
| 400 | Invalid paste ID format |
|
||||
| 404 | Paste not found |
|
||||
|
||||
---
|
||||
|
||||
### DELETE /{id}
|
||||
|
||||
Delete a paste. Requires authentication and ownership.
|
||||
|
||||
**Request:**
|
||||
```http
|
||||
DELETE /abc12345 HTTP/1.1
|
||||
Host: localhost:5000
|
||||
X-SSL-Client-SHA1: a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2
|
||||
```
|
||||
|
||||
**Response (200 OK):**
|
||||
```json
|
||||
{
|
||||
"message": "Paste deleted"
|
||||
}
|
||||
```
|
||||
|
||||
**Errors:**
|
||||
| Code | Description |
|
||||
|------|-------------|
|
||||
| 400 | Invalid paste ID format |
|
||||
| 401 | Authentication required |
|
||||
| 403 | Permission denied (not owner) |
|
||||
| 404 | Paste not found |
|
||||
|
||||
---
|
||||
|
||||
## MIME Type Detection
|
||||
|
||||
FlaskPaste automatically detects MIME types using:
|
||||
|
||||
1. **Magic byte signatures** (highest priority)
|
||||
- PNG: `\x89PNG\r\n\x1a\n`
|
||||
- JPEG: `\xff\xd8\xff`
|
||||
- GIF: `GIF87a` or `GIF89a`
|
||||
- WebP: `RIFF....WEBP`
|
||||
- ZIP: `PK\x03\x04`
|
||||
- PDF: `%PDF`
|
||||
- GZIP: `\x1f\x8b`
|
||||
|
||||
2. **Explicit Content-Type header** (if not generic)
|
||||
|
||||
3. **UTF-8 detection** (falls back to `text/plain`)
|
||||
|
||||
4. **Binary fallback** (`application/octet-stream`)
|
||||
|
||||
---
|
||||
|
||||
## Paste Expiry
|
||||
|
||||
Pastes expire based on last access time (default: 5 days).
|
||||
|
||||
- Every `GET /{id}` or `GET /{id}/raw` updates the last access timestamp
|
||||
- Cleanup runs automatically (hourly, throttled)
|
||||
- Configurable via `FLASKPASTE_EXPIRY` environment variable
|
||||
|
||||
---
|
||||
|
||||
## Error Response Format
|
||||
|
||||
All errors return JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": "Description of the error"
|
||||
}
|
||||
```
|
||||
|
||||
For size limit errors (413):
|
||||
```json
|
||||
{
|
||||
"error": "Paste too large",
|
||||
"size": 5000000,
|
||||
"max_size": 3145728,
|
||||
"authenticated": false
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Headers
|
||||
|
||||
All responses include the following security headers:
|
||||
|
||||
| Header | Value |
|
||||
|--------|-------|
|
||||
| `X-Content-Type-Options` | `nosniff` |
|
||||
| `X-Frame-Options` | `DENY` |
|
||||
| `X-XSS-Protection` | `1; mode=block` |
|
||||
| `Referrer-Policy` | `strict-origin-when-cross-origin` |
|
||||
| `Content-Security-Policy` | `default-src 'none'; frame-ancestors 'none'` |
|
||||
| `Permissions-Policy` | `geolocation=(), microphone=(), camera=()` |
|
||||
| `Strict-Transport-Security` | `max-age=31536000; includeSubDomains` |
|
||||
| `Cache-Control` | `no-store, no-cache, must-revalidate, private` |
|
||||
| `Pragma` | `no-cache` |
|
||||
|
||||
---
|
||||
|
||||
## Request Tracing
|
||||
|
||||
All requests include an `X-Request-ID` header for log correlation:
|
||||
|
||||
- If the client provides `X-Request-ID`, it is passed through
|
||||
- If not provided, a UUID is generated
|
||||
- The ID is echoed back in the response
|
||||
- All log entries include `[rid=<request-id>]`
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
# Client-provided ID
|
||||
curl -H "X-Request-ID: my-trace-123" https://paste.example.com/health
|
||||
|
||||
# Response includes:
|
||||
# X-Request-ID: my-trace-123
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Proxy Trust Validation
|
||||
|
||||
When `FLASKPASTE_PROXY_SECRET` is configured, the application validates that requests come from a trusted reverse proxy by checking the `X-Proxy-Secret` header.
|
||||
|
||||
This provides defense-in-depth against header spoofing if an attacker bypasses the reverse proxy.
|
||||
|
||||
**Configuration:**
|
||||
```bash
|
||||
export FLASKPASTE_PROXY_SECRET="your-secret-value"
|
||||
```
|
||||
|
||||
**Proxy Configuration (HAProxy):**
|
||||
```
|
||||
http-request set-header X-Proxy-Secret your-secret-value
|
||||
```
|
||||
|
||||
If the secret doesn't match, authentication headers (`X-SSL-Client-SHA1`) are ignored and the request is treated as anonymous.
|
||||
|
||||
If the secret doesn't match, authentication headers (`X-SSL-Client-SHA1`) are ignored and the request is treated as anonymous.
|
||||
305
documentation/deployment.md
Normal file
305
documentation/deployment.md
Normal file
@@ -0,0 +1,305 @@
|
||||
# FlaskPaste Deployment Guide
|
||||
|
||||
## Overview
|
||||
|
||||
FlaskPaste can be deployed in several ways:
|
||||
- Development server (for testing only)
|
||||
- WSGI server (Gunicorn, uWSGI)
|
||||
- Container (Podman/Docker)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.11+
|
||||
- SQLite 3
|
||||
- Reverse proxy for TLS termination (nginx, Apache, Caddy)
|
||||
|
||||
---
|
||||
|
||||
## Development Server
|
||||
|
||||
**For testing only - not for production!**
|
||||
|
||||
```bash
|
||||
# Setup
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Run
|
||||
python run.py
|
||||
# or
|
||||
flask --app app run --debug
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Production: WSGI Server
|
||||
|
||||
### Gunicorn
|
||||
|
||||
```bash
|
||||
pip install gunicorn
|
||||
|
||||
# Basic usage
|
||||
gunicorn -w 4 -b 0.0.0.0:5000 wsgi:app
|
||||
|
||||
# With Unix socket (recommended for nginx)
|
||||
gunicorn -w 4 --bind unix:/run/flaskpaste/gunicorn.sock wsgi:app
|
||||
|
||||
# Production settings
|
||||
gunicorn \
|
||||
--workers 4 \
|
||||
--bind unix:/run/flaskpaste/gunicorn.sock \
|
||||
--access-logfile /var/log/flaskpaste/access.log \
|
||||
--error-logfile /var/log/flaskpaste/error.log \
|
||||
--capture-output \
|
||||
wsgi:app
|
||||
```
|
||||
|
||||
### Systemd Service
|
||||
|
||||
Create `/etc/systemd/system/flaskpaste.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=FlaskPaste Pastebin Service
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=notify
|
||||
User=flaskpaste
|
||||
Group=flaskpaste
|
||||
RuntimeDirectory=flaskpaste
|
||||
WorkingDirectory=/opt/flaskpaste
|
||||
Environment="FLASK_ENV=production"
|
||||
Environment="FLASKPASTE_DB=/var/lib/flaskpaste/pastes.db"
|
||||
ExecStart=/opt/flaskpaste/venv/bin/gunicorn \
|
||||
--workers 4 \
|
||||
--bind unix:/run/flaskpaste/gunicorn.sock \
|
||||
wsgi:app
|
||||
ExecReload=/bin/kill -s HUP $MAINPID
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now flaskpaste
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Production: Container Deployment
|
||||
|
||||
### Build the Container
|
||||
|
||||
```bash
|
||||
# Using Podman
|
||||
podman build -t flaskpaste .
|
||||
|
||||
# Using Docker
|
||||
docker build -t flaskpaste .
|
||||
```
|
||||
|
||||
### Run the Container
|
||||
|
||||
```bash
|
||||
# Basic run
|
||||
podman run -d \
|
||||
--name flaskpaste \
|
||||
-p 5000:5000 \
|
||||
-v flaskpaste-data:/app/data \
|
||||
flaskpaste
|
||||
|
||||
# With environment configuration
|
||||
podman run -d \
|
||||
--name flaskpaste \
|
||||
-p 5000:5000 \
|
||||
-v flaskpaste-data:/app/data \
|
||||
-e FLASKPASTE_EXPIRY=604800 \
|
||||
-e FLASKPASTE_MAX_ANON=1048576 \
|
||||
flaskpaste
|
||||
```
|
||||
|
||||
### Podman Quadlet (systemd integration)
|
||||
|
||||
Create `/etc/containers/systemd/flaskpaste.container`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=FlaskPaste Container
|
||||
After=local-fs.target
|
||||
|
||||
[Container]
|
||||
Image=localhost/flaskpaste:latest
|
||||
ContainerName=flaskpaste
|
||||
PublishPort=5000:5000
|
||||
Volume=flaskpaste-data:/app/data:Z
|
||||
Environment=FLASK_ENV=production
|
||||
Environment=FLASKPASTE_EXPIRY=432000
|
||||
|
||||
[Service]
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target default.target
|
||||
```
|
||||
|
||||
```bash
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user start flaskpaste
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Reverse Proxy Configuration
|
||||
|
||||
### nginx
|
||||
|
||||
```nginx
|
||||
upstream flaskpaste {
|
||||
server unix:/run/flaskpaste/gunicorn.sock;
|
||||
# or for container: server 127.0.0.1:5000;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name paste.example.com;
|
||||
|
||||
ssl_certificate /etc/ssl/certs/paste.example.com.crt;
|
||||
ssl_certificate_key /etc/ssl/private/paste.example.com.key;
|
||||
|
||||
# Optional: Client certificate authentication
|
||||
ssl_client_certificate /etc/ssl/certs/ca.crt;
|
||||
ssl_verify_client optional;
|
||||
|
||||
# Size limit (should match FLASKPASTE_MAX_AUTH)
|
||||
client_max_body_size 50M;
|
||||
|
||||
location / {
|
||||
proxy_pass http://flaskpaste;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Pass client certificate fingerprint
|
||||
proxy_set_header X-SSL-Client-SHA1 $ssl_client_fingerprint;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Caddy
|
||||
|
||||
```caddyfile
|
||||
paste.example.com {
|
||||
reverse_proxy localhost:5000
|
||||
|
||||
# Client certificate auth (if needed)
|
||||
tls {
|
||||
client_auth {
|
||||
mode request
|
||||
trusted_ca_cert_file /etc/ssl/certs/ca.crt
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### HAProxy
|
||||
|
||||
```haproxy
|
||||
frontend https
|
||||
bind *:443 ssl crt /etc/haproxy/certs/
|
||||
|
||||
# Route to flaskpaste backend
|
||||
acl is_paste path_beg /paste
|
||||
use_backend backend-flaskpaste if is_paste
|
||||
|
||||
backend backend-flaskpaste
|
||||
mode http
|
||||
|
||||
# Strip /paste prefix before forwarding
|
||||
http-request replace-path /paste(.*) \1
|
||||
http-request set-path / if { path -m len 0 }
|
||||
|
||||
# Request tracing - generate X-Request-ID if not present
|
||||
http-request set-header X-Request-ID %[uuid()] unless { req.hdr(X-Request-ID) -m found }
|
||||
http-request set-var(txn.request_id) req.hdr(X-Request-ID)
|
||||
http-response set-header X-Request-ID %[var(txn.request_id)]
|
||||
|
||||
# Proxy trust secret - proves request came through HAProxy
|
||||
http-request set-header X-Proxy-Secret your-secret-value-here
|
||||
|
||||
# Pass client certificate fingerprint (if using mTLS)
|
||||
http-request set-header X-SSL-Client-SHA1 %[ssl_c_sha1,hex,lower]
|
||||
|
||||
server flaskpaste 127.0.0.1:5000 check
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Persistence
|
||||
|
||||
### Database Location
|
||||
|
||||
Default: `./data/pastes.db`
|
||||
|
||||
Configure via `FLASKPASTE_DB`:
|
||||
```bash
|
||||
export FLASKPASTE_DB=/var/lib/flaskpaste/pastes.db
|
||||
```
|
||||
|
||||
### Backup
|
||||
|
||||
```bash
|
||||
# SQLite online backup
|
||||
sqlite3 /var/lib/flaskpaste/pastes.db ".backup '/backup/pastes-$(date +%Y%m%d).db'"
|
||||
```
|
||||
|
||||
### Container Volume
|
||||
|
||||
```bash
|
||||
# Create named volume
|
||||
podman volume create flaskpaste-data
|
||||
|
||||
# Backup from volume
|
||||
podman run --rm \
|
||||
-v flaskpaste-data:/app/data:ro \
|
||||
-v ./backup:/backup \
|
||||
alpine cp /app/data/pastes.db /backup/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `FLASK_ENV` | `development` | Set to `production` in production |
|
||||
| `FLASKPASTE_DB` | `./data/pastes.db` | Database file path |
|
||||
| `FLASKPASTE_ID_LENGTH` | `12` | Paste ID length (hex chars) |
|
||||
| `FLASKPASTE_MAX_ANON` | `3145728` | Max anonymous paste (bytes) |
|
||||
| `FLASKPASTE_MAX_AUTH` | `52428800` | Max authenticated paste (bytes) |
|
||||
| `FLASKPASTE_EXPIRY` | `432000` | Paste expiry (seconds) |
|
||||
| `FLASKPASTE_PROXY_SECRET` | (empty) | Shared secret for proxy trust validation |
|
||||
|
||||
---
|
||||
|
||||
## Security Checklist
|
||||
|
||||
- [ ] Run behind reverse proxy with TLS
|
||||
- [ ] Use non-root user in containers
|
||||
- [ ] Set appropriate file permissions on database
|
||||
- [ ] Configure firewall (only expose reverse proxy)
|
||||
- [ ] Set `FLASK_ENV=production`
|
||||
- [ ] Configure client certificate auth (if needed)
|
||||
- [ ] Set `FLASKPASTE_PROXY_SECRET` for defense-in-depth
|
||||
- [ ] Configure proxy to send `X-Proxy-Secret` header
|
||||
- [ ] Configure proxy to pass/generate `X-Request-ID`
|
||||
- [ ] Set up log rotation
|
||||
- [ ] Enable automatic backups
|
||||
- [ ] Monitor disk usage (paste storage)
|
||||
Reference in New Issue
Block a user