claw/flaskpaste
1
0
Fork 1
forked from username/flaskpaste
Code Pull Requests Activity

docs: update project status for v1.2.0

Browse Source
This commit is contained in:
Username
2025-12-20 17:24:41 +01:00
parent 260d6e894d
commit 486bac1d85
4 changed files with 112 additions and 104 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
PROJECT.md
View File

@@ -18,8 +18,9 @@ Public pastebin services present risks:
A self-hosted pastebin API that:
- Stores pastes locally in SQLite
- Supports client certificate authentication via reverse proxy
- Provides optional built-in PKI for certificate management
- Automatically expires content based on access patterns
- Prevents abuse through content-hash deduplication
- Prevents abuse through content-hash deduplication and proof-of-work
- Serves text and binary content with proper MIME detection
- Runs behind any reverse proxy (nginx, HAProxy, Caddy)
@@ -32,6 +33,7 @@ A self-hosted pastebin API that:
│ Security │ Zero injection vulnerabilities
│ │ All OWASP headers implemented
│ │ Input validation on all endpoints
│ │ Security scanning in CI (bandit)
├────────────────────────────────┼────────────────────────────────────────────┤
│ Reliability │ SQLite ACID guarantees
│ │ Graceful degradation on errors
@@ -44,6 +46,7 @@ A self-hosted pastebin API that:
│ Operability │ Container-ready (Podman/Docker)
│ │ Gunicorn-compatible WSGI
│ │ Request tracing via X-Request-ID
│ │ Prometheus metrics endpoint
└────────────────────────────────┴────────────────────────────────────────────┘
```
@@ -55,6 +58,7 @@ A self-hosted pastebin API that:
- Text and binary content support
- Magic-byte MIME type detection
- Client certificate authentication (via proxy header)
- Built-in PKI (CA generation, certificate issuance, revocation)
- Configurable size limits (anon vs authenticated)
- Time-based expiry with access-touch semantics
- Content-hash deduplication for abuse prevention
@@ -68,6 +72,7 @@ A self-hosted pastebin API that:
- Request tracing and structured logging
- Container deployment support
- SQLite storage
- CI/CD with security scanning
### Out of Scope
@@ -90,7 +95,7 @@ A self-hosted pastebin API that:
## Assumptions
- Deployment behind a TLS-terminating reverse proxy
- Client certificates managed externally (PKI, mTLS)
- Client certificates managed externally or via built-in PKI
- Operators have container runtime (Podman/Docker) or Python venv
- SQLite performance sufficient for expected load
@@ -104,14 +109,17 @@ A self-hosted pastebin API that:
│ Database │ SQLite 3 (built-in)
│ WSGI Server │ Gunicorn (production)
│ Container │ Podman / Docker
│ Testing │ pytest, pytest-cov
│ Testing │ pytest
│ Linting │ ruff, mypy
│ Security │ bandit, pip-audit
│ CI/CD │ Gitea Actions
│ Python │ 3.11+
└─────────────────┴──────────────────────────────────────────────────────────┘
```
## Current Status
**Version:** 1.1.0
**Version:** 1.2.0
```
┌─────────────────────────────────┬────────────────────────────────────────────┐
@@ -121,6 +129,7 @@ A self-hosted pastebin API that:
│ Binary content support │ Complete
│ MIME detection │ Complete
│ Client cert authentication │ Complete
│ Built-in PKI │ Complete
│ Size limits │ Complete
│ Paste expiry │ Complete
│ Content-hash deduplication │ Complete
@@ -134,6 +143,8 @@ A self-hosted pastebin API that:
│ Security headers │ Complete
│ Request tracing │ Complete
│ Container deployment │ Complete
│ Test suite │ 113 tests passing
│ Security tooling │ Complete
│ CI/CD pipeline │ Complete
│ Test suite │ 147 tests passing
└─────────────────────────────────┴────────────────────────────────────────────┘
```

136
ROADMAP.md
View File

@@ -2,12 +2,13 @@
## Current State
FlaskPaste v1.1.0 is deployed and feature-complete for its core mission: a secure, minimal pastebin API with zero-knowledge encryption support.
FlaskPaste v1.2.0 is deployed with PKI integration and comprehensive security tooling.
**Implemented:**
- Full REST API (CRUD operations)
- Binary content support with magic-byte MIME detection
- Client certificate authentication
- Minimal PKI (CA generation, certificate issuance, revocation)
- Content-hash deduplication (abuse prevention)
- Proof-of-work spam prevention
- Entropy enforcement (require encrypted uploads)
@@ -15,116 +16,102 @@ FlaskPaste v1.1.0 is deployed and feature-complete for its core mission: a secur
- URL prefix support for reverse proxy deployments
- /client endpoint for CLI distribution
- Automatic paste expiry
- Burn-after-read pastes
- Custom expiry per paste
- Security headers and request tracing
- Container deployment support
- Comprehensive test suite (98 tests)
- Security tooling (ruff, bandit, mypy, pip-audit)
- CI/CD pipeline with lint, security, and test jobs
- Comprehensive test suite (147 tests)
## Phase 1: Hardening (Complete)
Focus: Production readiness and operational excellence.
```
┌───┬─────────────────────────────────────┬────────────────────────────────────┐
│ # │ Milestone │ Status
├───┼─────────────────────────────────────┼────────────────────────────────────┤
│ 1 │ Abuse prevention (dedup) │ Done
│ 2 │ Security headers complete │ Done
│ 3 │ Request tracing (X-Request-ID) │ Done
│ 4 │ Proxy trust validation │ Done
│ 5 │ Proof-of-work spam prevention │ Done
│ 6 │ Entropy enforcement │ Done
│ 7 │ Test coverage > 90% │ Done (98 tests)
│ 8 │ Documentation complete │ Done
└───┴─────────────────────────────────────┴────────────────────────────────────┘
┌───┬─────────────────────────────────┬────────────────────────────────────┐
│ # │ Milestone │ Status
├───┼─────────────────────────────────┼────────────────────────────────────┤
│ 1 │ Abuse prevention (dedup) │ Done
│ 2 │ Security headers complete │ Done
│ 3 │ Request tracing (X-Request-ID) │ Done
│ 4 │ Proxy trust validation │ Done
│ 5 │ Proof-of-work spam prevention │ Done
│ 6 │ Entropy enforcement │ Done
│ 7 │ Test coverage > 90% │ Done (147 tests)
│ 8 │ Documentation complete │ Done
└───┴─────────────────────────────────┴────────────────────────────────────┘
```
## Phase 2: Operations
## Phase 2: Operations (Complete)
Focus: Deployment, monitoring, and maintenance tooling.
```
┌───┬─────────────────────────────────────┬────────────────────────────────────┐
│ # │ Milestone │ Dependencies
├───┼─────────────────────────────────────┼────────────────────────────────────┤
│ 1 │ Prometheus metrics endpoint │ None
│ 2 │ Structured JSON logging │ None
│ 3 │ Admin API (stats, cleanup) │ Auth improvements
│ 4 │ Ansible deployment role │ None
│ 5 │ CI/CD pipeline │ Container registry access
└───┴─────────────────────────────────────┴────────────────────────────────────┘
┌───┬─────────────────────────────────┬────────────────────────────────────┐
│ # │ Milestone │ Status
├───┼─────────────────────────────────┼────────────────────────────────────┤
│ 1 │ Prometheus metrics endpoint │ Done (prometheus-flask-exporter)
│ 2 │ Structured JSON logging │ Done (production mode)
│ 3 │ Security tooling (lint/scan) │ Done (ruff, bandit, mypy)
│ 4 │ CI/CD pipeline │ Done (Gitea Actions)
│ 5 │ Multi-stage Containerfile │ Done
└───┴─────────────────────────────────┴────────────────────────────────────┘
```
### Prometheus Metrics
Expose `/metrics` endpoint with:
- `flaskpaste_pastes_total` (counter)
- `flaskpaste_pastes_created` (counter)
- `flaskpaste_pastes_deleted` (counter)
- `flaskpaste_pastes_expired` (counter)
- `flaskpaste_storage_bytes` (gauge)
- `flaskpaste_request_duration_seconds` (histogram)
### Structured Logging
Replace text logs with JSON format:
- Timestamp, level, message, request_id
- Consistent field names across all log entries
- Compatible with log aggregation (Loki, ELK)
## Phase 3: Features (Complete)
Focus: User-requested enhancements within scope.
```
┌───┬─────────────────────────────────────┬────────────────────────────────────┐
│ # │ Feature │ Status
├───┼─────────────────────────────────────┼────────────────────────────────────┤
│ 1 │ E2E encryption (client-side) │ Done (CLI -e flag, zero-knowledge)
│ 2 │ URL prefix support │ Done
│ 3 │ Custom expiry per paste │ Done (X-Expiry header)
│ 4 │ Burn-after-read option │ Done (X-Burn-After-Read header)
│ 5 │ Paste password protection │ Planned
└───┴─────────────────────────────────────┴────────────────────────────────────┘
┌───┬─────────────────────────────────┬────────────────────────────────────┐
│ # │ Feature │ Status
├───┼─────────────────────────────────┼────────────────────────────────────┤
│ 1 │ E2E encryption (client-side) │ Done (CLI -e flag, zero-knowledge)
│ 2 │ URL prefix support │ Done
│ 3 │ Custom expiry per paste │ Done (X-Expiry header)
│ 4 │ Burn-after-read option │ Done (X-Burn-After-Read header)
│ 5 │ Minimal PKI (CA + issuance) │ Done
└───┴─────────────────────────────────┴────────────────────────────────────┘
```
### Burn-After-Read
### PKI Features
Single-access pastes that delete after first retrieval:
- `POST /` with `X-Burn-After-Read: true` header
- Paste deleted after first `GET /<id>/raw`
- Metadata `GET /<id>` does not trigger burn
Integrated certificate authority for mTLS:
- `POST /pki/ca` - Generate CA (first-run bootstrap)
- `GET /pki/status` - CA status and fingerprint
- `GET /pki/ca.crt` - Download CA certificate
- `POST /pki/issue` - Issue client certificate
- `POST /pki/revoke/<serial>` - Revoke certificate
- CLI: `fpaste pki status`, `fpaste pki issue`, `fpaste pki revoke`
### Custom Expiry
Allow per-paste expiry override:
- `POST /` with `X-Expiry: 3600` header (seconds)
- Capped at server maximum (e.g., 30 days)
- Default unchanged for pastes without header
## Phase 4: Ecosystem
## Phase 4: Ecosystem (In Progress)
Focus: Integration with external systems.
```
┌───┬─────────────────────────────────────┬────────────────────────────────────┐
│ # │ Integration │ Status
├───┼─────────────────────────────────────┼────────────────────────────────────┤
│ 1 │ CLI client (fpaste) │ Done (with E2E encryption)
│ 2 │ /client endpoint │ Done (downloadable CLI)
│ 3 │ Neovim/Vim plugin │ Planned
│ 4 │ Shell aliases/functions │ Planned
│ 5 │ Webhook notifications │ Planned
└───┴─────────────────────────────────────┴────────────────────────────────────┘
┌───┬─────────────────────────────────┬────────────────────────────────────┐
│ # │ Integration │ Status
├───┼─────────────────────────────────┼────────────────────────────────────┤
│ 1 │ CLI client (fpaste) │ Done (with E2E + PKI)
│ 2 │ /client endpoint │ Done (downloadable CLI)
│ 3 │ Ansible deployment role │ Planned
│ 4 │ Kubernetes manifests │ Planned
│ 5 │ Shell aliases/functions │ Planned
└───┴─────────────────────────────────┴────────────────────────────────────┘
```
### CLI Client (Complete)
Standalone Python CLI with encryption support:
Standalone Python CLI with encryption and PKI support:
- `fpaste create file.txt` - Create paste from file
- `fpaste create -e file.txt` - Create encrypted paste (E2E)
- `fpaste get <id>` - Get paste (auto-decrypts with URL fragment key)
- `fpaste delete <id>` - Delete paste
- `fpaste info` - Show server info
- `fpaste pki status` - Show PKI status
- `fpaste pki issue -n "name"` - Request client certificate
- `fpaste pki revoke <serial>` - Revoke certificate
- Config file for server URL and cert fingerprint
- Downloadable via `curl https://server/client > fpaste`
@@ -154,6 +141,9 @@ These features will not be implemented:
| 2024-12 | Burn-after-read | Single-use pastes for sensitive data
| 2024-12 | Custom expiry | Per-paste TTL override
| 2024-12 | Multi-stage Containerfile | Smaller production images
| 2024-12 | Minimal PKI | Self-contained mTLS without external CA
| 2024-12 | Security tooling (ruff/bandit) | Code quality and security scanning
| 2024-12 | CI/CD with job dependencies | Tests wait for lint to pass
## Review Schedule

33
TASKLIST.md
View File

@@ -4,38 +4,47 @@ Prioritized, actionable tasks. Each task is small and completable in one session
---
## Priority 1: Operations
## Priority 1: Ecosystem
| Status | Task
|--------|--------------------------------------------------------------
| ☐ | Add /metrics endpoint skeleton (for future Prometheus)
| ☐ | Add structured logging option via environment variable
| ☐ | Create Ansible deployment role
| ☐ | Add Kubernetes manifests (Deployment, Service, ConfigMap)
| ☐ | Add systemd service unit example
## Priority 2: Features
| Status | Task
|--------|--------------------------------------------------------------
| ☐ | Add paste listing for authenticated users
| ☐ | Add paste password protection
| ☐ | Add rate limit headers (X-RateLimit-*)
## Priority 3: Documentation
| Status | Task
|--------|--------------------------------------------------------------
| ☐ | Add deployment examples (Kubernetes, systemd)
| ☐ | Create CONTRIBUTING.md with development setup
## Priority 4: Testing
## Priority 3: Quality
| Status | Task
|--------|--------------------------------------------------------------
| ☐ | Fix mypy type errors (currently ignored)
| ☐ | Add test for concurrent identical submissions
| ☐ | Add integration tests for container deployment
## Priority 4: Documentation
| Status | Task
|--------|--------------------------------------------------------------
| ☐ | Create CONTRIBUTING.md with development setup
| ☐ | Add PKI usage examples to documentation
## Completed
| Date | Task
|------------|--------------------------------------------------------------
| 2024-12 | Implement minimal PKI (CA, issuance, revocation)
| 2024-12 | Add security tooling (ruff, bandit, mypy, pip-audit)
| 2024-12 | Create Makefile with dev workflow targets
| 2024-12 | Setup CI/CD pipeline (Gitea Actions)
| 2024-12 | Fix all ruff lint issues
| 2024-12 | Optimize CI workflow (concurrency, job deps)
| 2024-12 | Add PKI commands to CLI (status, issue, revoke)
| 2024-12 | Implement burn-after-read option
| 2024-12 | Implement custom expiry per paste
| 2024-12 | Optimize Containerfile with multi-stage build

26
TODO.md
View File

@@ -6,34 +6,33 @@ Unstructured intake buffer for ideas, issues, and observations. Items here are r
## Ideas
- Prometheus metrics endpoint (`/metrics`) for monitoring integration
- Structured JSON logging for log aggregation compatibility
- Burn-after-read paste option
- Custom expiry header for per-paste TTL
- Rate limit headers in responses (X-RateLimit-*)
- Paste compression for large text content
- ETag support for conditional requests
- Paste listing for authenticated users (their own pastes only)
- Neovim/Vim plugin for editor integration
- Webhook notifications for paste events
- Certificate renewal reminder in CLI
- Admin endpoint for CA key rotation
## Observations
- Abuse prevention uses content-hash dedup + PoW + entropy enforcement
- SQLite WAL mode could improve concurrent read performance
- Container image size could be reduced with multi-stage build
- E2E encryption in CLI uses cryptography package (optional dependency)
- Entropy check has size threshold to avoid false positives on small data
- PKI uses AES-256-GCM for CA private key encryption (PBKDF2 key derivation)
- SHA1 fingerprints are X.509 standard, not security-relevant (usedforsecurity=False)
- Revoked certificates are soft-deleted (status tracked, not removed)
- CI pipeline: lint runs parallel with security, tests wait for lint
- Ruff replaces flake8/isort/pyupgrade with single fast tool
- Bandit configured for medium+ severity only (-ll flag)
## Questions
- Should expired paste cleanup run in-process or via external cron?
- Is SQLite sufficient for anticipated load, or plan for PostgreSQL?
- Should burn-after-read pastes show in metadata before burn?
- Password-protected pastes: derive key from password or store hash?
- Certificate renewal: reissue with same CN or require new request?
- Should revoked certs be purged after grace period?
## Debt
- Mypy has pre-existing type errors (runs with --ignore-missing-imports)
- No integration tests for container deployment
- Missing test for concurrent paste creation
- Could add more deployment examples (Kubernetes, systemd)
@@ -41,8 +40,7 @@ Unstructured intake buffer for ideas, issues, and observations. Items here are r
## External Dependencies
- Consider adding `python-magic` for better MIME detection (currently magic bytes only)
- Evaluate `structlog` for structured logging when implemented
- Look into `prometheus-flask-exporter` for metrics
- cryptography package required for PKI features (optional otherwise)
---