username/flaskpaste
1
0
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add url shortener documentation
Some checks failed
CI / Lint & Format (push) Failing after 29s
Details
CI / Unit Tests (push) Has been skipped
Details
CI / Memory Leak Check (push) Has been skipped
Details
CI / Fuzz Testing (push) Has been skipped
Details
CI / SBOM Generation (push) Has been skipped
Details
CI / Security Scan (push) Successful in 33s
Details
CI / Security Tests (push) Has been skipped
Details
CI / Advanced Security Tests (push) Has been skipped
Details
CI / Build & Push Image (push) Has been skipped
Details
CI / Harbor Vulnerability Scan (push) Has been skipped
Details

Browse Source
This commit is contained in:
Username
2026-02-16 20:56:55 +01:00
parent 75a9bf56d9
commit 2679bc8e69
6 changed files with 268 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
PROJECT.md
View File

@@ -67,6 +67,7 @@ A self-hosted pastebin API that:
- Client-side E2E encryption (CLI)
- Burn-after-read pastes
- Custom expiry per paste
- URL shortener with open redirect prevention
- URL prefix for reverse proxy deployments
- Security headers (HSTS, CSP, X-Frame-Options, etc.)
- Request tracing and structured logging
@@ -118,7 +119,7 @@ A self-hosted pastebin API that:
## Current Status
**Version:** 1.5.0
**Version:** 1.6.0
```
┌─────────────────────────────────┬────────────────────────────────────────────┐
@@ -151,8 +152,9 @@ A self-hosted pastebin API that:
│ Public certificate registration │ Complete
│ CLI register command │ Complete
│ systemd deployment │ Complete (security-hardened)
│ Test suite │ 301 tests passing
│ Test suite │ 346 tests passing
│ Kubernetes deployment │ Complete (k3s, NodePort :30500)
│ Harbor registry integration │ Complete (CI/CD + Trivy scanning)
│ URL shortener │ Complete (8-char base62, redirect, info)
└─────────────────────────────────┴────────────────────────────────────────────┘
```

23
README.md
View File

@@ -22,6 +22,7 @@ A lightweight, secure pastebin REST API built with Flask.
- **Security headers** - HSTS, CSP, X-Frame-Options, X-Content-Type-Options
- **CLI client** - Standalone `fpaste` tool with encryption support
- **Request tracing** - X-Request-ID for log correlation
- **URL shortener** - `/s/` endpoints for creating, resolving, and managing short URLs
- **Audit logging** - PKI certificate lifecycle events (issue, revoke, auth failure)
- **Observability** - Request duration metrics via Prometheus histogram
- **Minimal dependencies** - Flask + SQLite, optional cryptography for CLI
@@ -57,6 +58,11 @@ python run.py
| `GET /pastes` | List user's pastes (requires auth) |
| `GET /register/challenge` | Get PoW challenge for registration |
| `POST /register` | Register and get client certificate |
| `POST /s` | Create short URL (PoW required) |
| `GET /s` | List your short URLs (requires auth) |
| `GET /s/<id>` | Redirect to target URL (302) |
| `GET /s/<id>/info` | Short URL metadata |
| `DELETE /s/<id>` | Delete short URL (requires auth) |
| `GET /pki` | PKI status and CA info |
| `GET /pki/ca.crt` | Download CA certificate |
@@ -102,6 +108,18 @@ curl -H "X-SSL-Client-SHA1: <your-cert-fingerprint>" \
http://localhost:5000/pastes
```
### Create a short URL
```bash
curl -X POST -H "Content-Type: application/json" \
-d '{"url":"https://example.com/long/path"}' \
http://localhost:5000/s
```
### Follow a short URL
```bash
curl -L http://localhost:5000/s/AbCdEfGh
```
## CLI Client
A standalone command-line client `fpaste` is included. For E2E encryption, install the optional `cryptography` package.
@@ -280,6 +298,8 @@ Configuration via environment variables:
| `FLASKPASTE_RATE_WINDOW` | `60` | Rate limit window (seconds) |
| `FLASKPASTE_RATE_MAX` | `10` | Max requests per window (anon) |
| `FLASKPASTE_RATE_AUTH_MULT` | `5` | Multiplier for authenticated users |
| `FLASKPASTE_SHORT_ID_LENGTH` | `8` | Short URL ID length (base62 characters) |
| `FLASKPASTE_SHORT_URL_MAX` | `2048` | Maximum target URL length |
| `FLASKPASTE_URL_PREFIX` | (empty) | URL prefix for reverse proxy deployments |
| `FLASKPASTE_MIN_ENTROPY` | `0` | Min entropy bits/byte (0=disabled, 6.0=require encryption) |
| `FLASKPASTE_MIN_ENTROPY_SIZE` | `256` | Only check entropy for content >= this size |
@@ -419,7 +439,7 @@ flaskpaste/
│ └── api/
│ ├── __init__.py # Blueprint setup
│ └── routes.py # API endpoints
├── tests/ # Test suite (356 tests)
├── tests/ # Test suite (346+ tests)
├── data/ # SQLite database
├── run.py # Development server
├── wsgi.py # Production WSGI entry
@@ -444,6 +464,7 @@ flaskpaste/
- **Security headers** - HSTS, CSP, X-Frame-Options, X-Content-Type-Options
- **Proof-of-work** - Computational puzzles prevent automated spam
- **Rate limiting** - Per-IP throttling with X-RateLimit-* headers
- **Open redirect prevention** - URL shortener allows only http/https schemes with valid host
- **Request tracing** - X-Request-ID for log correlation
- **PKI support** - Built-in CA for client certificate issuance
- **Audit logging** - PKI certificate events for compliance and forensics

19
ROADMAP.md
View File

@@ -29,7 +29,8 @@ FlaskPaste v1.5.1 is deployed with comprehensive security hardening and abuse pr
- CLI with list, search, update, export commands
- Public certificate registration (PoW-protected)
- CLI register command for certificate enrollment
- Comprehensive test suite (356 tests)
- URL shortener (create, redirect, info, delete, list)
- Comprehensive test suite (346 tests)
- Complete security pentest remediation (15 items)
- PKI audit logging (certificate lifecycle events)
- Request duration metrics (Prometheus histogram)
@@ -49,7 +50,7 @@ Focus: Production readiness and operational excellence.
│ 4 │ Proxy trust validation │ Done
│ 5 │ Proof-of-work spam prevention │ Done
│ 6 │ Entropy enforcement │ Done
│ 7 │ Test coverage > 90% │ Done (301 tests)
│ 7 │ Test coverage > 90% │ Done (346 tests)
│ 8 │ Documentation complete │ Done
└───┴─────────────────────────────────┴────────────────────────────────────┘
```
@@ -86,9 +87,21 @@ Focus: User-requested enhancements within scope.
│ 6 │ Anti-flood (dynamic PoW) │ Done (v1.4.0)
│ 7 │ IP-based rate limiting │ Done (v1.4.0)
│ 8 │ Scheduled cleanup │ Done (v1.4.0)
│ 9 │ URL shortener │ Done (v1.6.0)
└───┴─────────────────────────────────┴────────────────────────────────────┘
```
### URL Shortener (v1.6.0)
Short URL creation, redirect, metadata, and management:
- `POST /s` - Create short URL (PoW + rate limit)
- `GET /s` - List own short URLs (auth required)
- `GET /s/<id>` - 302 redirect to target
- `GET /s/<id>/info` - JSON metadata (target, clicks, expiry)
- `DELETE /s/<id>` - Delete (owner only)
- Open redirect prevention (http/https only, netloc required)
- 8-char base62 IDs (visually distinct from paste hex IDs)
### Anti-Flood System (v1.4.0)
Dynamic proof-of-work difficulty that increases under abuse:
@@ -193,6 +206,8 @@ These features will not be implemented:
| 2024-12 | Pentest remediation complete | 15 security hardening items from formal review
| 2024-12 | Enhanced CI security | SBOM generation, dedicated security-tests job
| 2025-01 | CI/CD image build/push | Auto-build on main, push to Harbor registry
| 2026-02 | URL shortener | /s/ prefix avoids paste ID collision; base62 IDs
| 2026-02 | Open redirect prevention | http/https only, netloc required, 2048 byte limit
## Review Schedule

27
SECURITY.md
View File

@@ -92,6 +92,25 @@ FLASKPASTE_MIN_ENTROPY=6.0 # Bits per byte (encrypted ~7.5-8.0)
FLASKPASTE_MIN_ENTROPY_SIZE=256 # Only check content >= this size
```
### URL Shortener Security
**Open Redirect Prevention**
Short URL creation validates target URLs:
- Only `http` and `https` schemes allowed (rejects `javascript:`, `data:`, `ftp:`, `file:`)
- Network location (hostname) required — rejects scheme-only URLs
- Maximum URL length: 2048 bytes
- Short IDs: 8-char base62 (`[a-zA-Z0-9]`) with `secrets.choice()` for unpredictability
- Redirect responses include `Cache-Control: no-store, no-cache` to prevent caching
**Access Controls**
- Creation: rate-limited + proof-of-work (same as paste creation)
- Redirect: lookup rate limiting prevents enumeration
- Deletion: owner authentication required
- Listing: authentication required, shows only own URLs
### Security Headers
All responses include:
@@ -120,6 +139,12 @@ All requests receive `X-Request-ID` header for log correlation and debugging. Pa
- Configurable length (default 12 characters)
- Validated on all endpoints
### Short URL IDs
- Base62 only (`[a-zA-Z0-9]+`)
- 8 characters (configurable via `FLASKPASTE_SHORT_ID_LENGTH`)
- Validated on all `/s/` endpoints
### MIME Types
- Magic byte detection for binary formats
@@ -245,6 +270,7 @@ Security fixes are released as soon as possible. Subscribe to repository release
- Authentication bypass
- Information disclosure
- Denial of service (application-level)
- Open redirect via URL shortener
### Out of Scope
@@ -258,6 +284,7 @@ Security fixes are released as soon as possible. Subscribe to repository release
| Version | Security Changes |
|---------|------------------|
| 1.6.0 | URL shortener with open redirect prevention, scheme allowlist, target URL validation |
| 1.5.0 | Pentest remediation (15 items): timing attack prevention, serial collision detection, lookup rate limiting, content hash locking, anti-flood memory limits, CLI path validation, SSL hostname verification, config permission checks |
| 1.4.0 | Anti-flood dynamic PoW, IP-based rate limiting, audit logging |
| 1.2.0 | Password protection with PBKDF2, code modernization |

1
TASKLIST.md
View File

@@ -14,6 +14,7 @@ Prioritized, actionable tasks. Each task is small and completable in one session
| Date | Task
|------------|--------------------------------------------------------------
| 2026-02 | Add URL shortener (create, redirect, info, delete, list)
| 2025-01 | Add CI/CD image build and push to Harbor
| 2025-01 | Add Kubernetes manifests (Deployment, Service, ConfigMap)
| 2024-12 | Add PKI usage examples (documentation/pki.md)

197
documentation/api.md
View File

@@ -459,6 +459,203 @@ X-SSL-Client-SHA1: a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2
### POST /s
Create a new short URL. Requires proof-of-work and respects rate limits.
**Request (JSON):**
```http
POST /s HTTP/1.1
Host: localhost:5000
Content-Type: application/json
X-PoW-Token: <token>
X-PoW-Solution: <solution>
```
**Request (Raw):**
```http
POST /s HTTP/1.1
Host: localhost:5000
Content-Type: text/plain
X-PoW-Token: <token>
X-PoW-Solution: <solution>
```
**Optional Headers:**
| Header | Description |
|--------|-------------|
| `X-Expiry` | Custom expiry in seconds |
| `X-SSL-Client-SHA1` | Client certificate fingerprint (for ownership) |
**Response (201 Created):**
```json
{
"id": "AbCdEfGh",
"url": "/s/AbCdEfGh",
"target_url": "https://example.com/some/long/path?query=value",
"created_at": 1700000000,
"owner": "a1b2c3d4...",
"expires_at": 1700003600
}
```
**Errors:**
| Code | Description |
|------|-------------|
| 400 | No URL provided |
| 400 | Invalid URL scheme (only http/https allowed) |
| 400 | Invalid URL: missing host |
| 400 | URL too long (max 2048 bytes) |
| 400 | Proof-of-work required/failed |
| 429 | Rate limit or duplicate URL limit exceeded |
**Security:**
- Only `http` and `https` schemes are accepted (prevents `javascript:`, `data:`, `file:` etc.)
- URLs must have a valid network location (host)
- Maximum URL length: 2048 bytes (configurable via `FLASKPASTE_SHORT_URL_MAX`)
---
### GET /s
List short URLs owned by the authenticated user.
**Request:**
```http
GET /s HTTP/1.1
Host: localhost:5000
X-SSL-Client-SHA1: a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2
```
**Query Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `limit` | int | Maximum results (default: 50, max: 200) |
| `offset` | int | Pagination offset (default: 0) |
**Response (200 OK):**
```json
{
"urls": [
{
"id": "AbCdEfGh",
"target_url": "https://example.com",
"created_at": 1700000000,
"access_count": 42,
"url": "/s/AbCdEfGh"
}
],
"count": 1,
"total": 1,
"limit": 50,
"offset": 0
}
```
**Errors:**
| Code | Description |
|------|-------------|
| 401 | Authentication required |
---
### GET /s/{id}
### HEAD /s/{id}
Redirect to the target URL. Returns HTTP 302 with `Location` header.
**Request:**
```http
GET /s/AbCdEfGh HTTP/1.1
Host: localhost:5000
```
**Response (302 Found):**
```http
HTTP/1.1 302 Found
Location: https://example.com/some/long/path
Cache-Control: no-store, no-cache, must-revalidate, private
```
Each access increments the short URL's access counter.
**Errors:**
| Code | Description |
|------|-------------|
| 400 | Invalid short URL ID format |
| 404 | Short URL not found or expired |
---
### GET /s/{id}/info
Retrieve short URL metadata without incrementing the access counter.
**Request:**
```http
GET /s/AbCdEfGh/info HTTP/1.1
Host: localhost:5000
```
**Response (200 OK):**
```json
{
"id": "AbCdEfGh",
"target_url": "https://example.com/some/long/path",
"created_at": 1700000000,
"last_accessed": 1700001000,
"access_count": 42,
"url": "/s/AbCdEfGh",
"owner": "a1b2c3d4...",
"expires_at": 1700086400
}
```
**Errors:**
| Code | Description |
|------|-------------|
| 400 | Invalid short URL ID format |
| 404 | Short URL not found or expired |
---
### DELETE /s/{id}
Delete a short URL. Requires authentication and ownership (or admin rights).
**Request:**
```http
DELETE /s/AbCdEfGh HTTP/1.1
Host: localhost:5000
X-SSL-Client-SHA1: a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2
```
**Response (200 OK):**
```json
{
"message": "Short URL deleted"
}
```
**Errors:**
| Code | Description |
|------|-------------|
| 400 | Invalid short URL ID format |
| 401 | Authentication required |
| 403 | Permission denied (not owner or admin) |
| 404 | Short URL not found |
---
## MIME Type Detection
FlaskPaste automatically detects MIME types using:
1. **Magic byte signatures** (highest priority)
| Category | Formats |
|----------|---------|
| Images | PNG, JPEG, GIF, WebP, BMP, TIFF, ICO |
| Video | MP4, WebM, FLV, Matroska |
| Audio | MP3, FLAC, OGG |