docs: update for simplified MIME detection (v1.5.1)

ROADMAP.md

@@ -2,11 +2,11 @@
 
 ## Current State
 
-FlaskPaste v1.5.0 is deployed with comprehensive security hardening and abuse prevention.
+FlaskPaste v1.5.1 is deployed with comprehensive security hardening and abuse prevention.
 
 **Implemented:**
 - Full REST API (CRUD operations)
-- Binary content support with magic-byte MIME detection
+- Binary content support with text/binary MIME detection
 - Client certificate authentication
 - Minimal PKI (CA generation, certificate issuance, revocation)
 - Content-hash deduplication (abuse prevention)

TODO.md

@@ -12,7 +12,7 @@ Unstructured intake buffer for ideas, issues, and observations. Items here are r
   - Design: compress-then-encrypt only (not compress-only)
   - Compressed data has high entropy → bypasses entropy enforcement
   - Must enforce encryption when compression enabled (CLI-side)
-  - Server detects compression formats via magic bytes (REQUIRE_BINARY)
+  - Server rejects plaintext via REQUIRE_BINARY (UTF-8 detection)
 - ETag support for conditional requests
 - Neovim/Vim plugin for editor integration
 - Webhook notifications for paste events
@@ -53,8 +53,8 @@ Unstructured intake buffer for ideas, issues, and observations. Items here are r
 
 ## External Dependencies
 
-- Consider adding `python-magic` for better MIME detection (currently magic bytes only)
 - cryptography package required for PKI features (optional otherwise)
+- For full MIME detection, consider `filetype` library (currently text/binary only)
 
 ---
 

documentation/api.md

@@ -707,17 +707,16 @@ export FLASKPASTE_MIN_ENTROPY_SIZE=256 # Only check content >= this size (defaul
 - Plaintext (valid UTF-8) is rejected with 400
 - Only binary content (invalid UTF-8) is allowed
 
-
+**Configuration:**
 ```bash
 export FLASKPASTE_REQUIRE_BINARY=1  # Reject plaintext (0=disabled)
-export FLASKPASTE_REQUIRE_BINARY=1  # Reject recognizable formats (0=disabled)
 ```
-
+
-**Response (400 Bad Request):**
+**Response (400 Bad Request):**
 ```json
 {
   "error": "Recognizable format not allowed",
-  "detected": "text/plain",
+  "detected": "text/plain",
   "hint": "Encrypt content before uploading (fpaste encrypts by default)"
 }
 ```
@@ -729,16 +728,11 @@ export FLASKPASTE_REQUIRE_BINARY=1  # Reject recognizable formats (0=disabled)
 | Binary | Valid UTF-8 text | Reject plaintext |
 
 Use both together for maximum encryption enforcement:
-|--------|---------|-----------------|
-| Entropy | Random-looking data | Compressed files pass |
-| Binary | No magic bytes + invalid UTF-8 | Minimal |
-
-Use both together for maximum encryption enforcement:
 ```bash
-export FLASKPASTE_REQUIRE_BINARY=1
+export FLASKPASTE_REQUIRE_BINARY=1
-export FLASKPASTE_MIN_ENTROPY=6.0
+export FLASKPASTE_MIN_ENTROPY=6.0
-export FLASKPASTE_MIN_SIZE=64
+export FLASKPASTE_MIN_SIZE=64
-```
+```
 
 ---
 

documentation/mime-security-assessment.md

@@ -1,5 +1,10 @@
 # MIME Detection Security Assessment
 
+> **Note (v1.5.1):** Magic byte detection has been simplified to UTF-8 validation only.
+> Content is now classified as `text/plain` (valid UTF-8) or `application/octet-stream` (binary).
+> Security headers (nosniff, CSP) provide the primary defense against MIME confusion attacks.
+> This document is retained for historical reference.
+
 Penetration testing of FlaskPaste's magic byte-based MIME detection system.
 
 ---

documentation/security-testing-status.md

@@ -328,14 +328,13 @@ DEDUP_MAX = 3                       # Max duplicates allowed
 | X-Content-Type-Options | nosniff | Yes |
 | Content-Security-Policy | default-src 'none' | Yes |
 | X-Frame-Options | DENY | Yes |
-| Magic byte detection | First 16 bytes, 45 signatures | Yes |
+| MIME detection | UTF-8 validation (text/binary) | Yes |
 | Input sanitization | Werkzeug header handling | Yes |
 | SQL injection prevention | SQLAlchemy parameterized queries | Yes |
 | SSTI prevention | No user content in templates | Yes |
 | Path traversal prevention | ID validation regex | Yes |
 | Constant-time password check | PBKDF2 600k iterations | Yes |
 | Burn-after-read race condition | HEAD triggers deletion | Yes |
-| RIFF container detection | Subtype check (WEBP/AVI/WAVE) | Yes |
 | Clipboard command injection | Trusted path validation | Yes |
 | Memory exhaustion prevention | Max entries on all dicts | Yes |
 | Race condition protection | Threading locks on counters | Yes |

documentation/threat-model.md

@@ -191,28 +191,27 @@ User Input Flow:
 
 ## MIME Detection Security
 
-Content is detected by magic bytes, not user-supplied Content-Type:
+Content is detected by UTF-8 validation (text vs binary):
 
 ```
-User uploads "image.png" with PHP payload
+User uploads content
          |
          v
-[Magic byte detection] --> Not PNG magic --> text/plain
+[UTF-8 validation] --> Valid UTF-8 --> text/plain
-         |
+         |              Invalid --> application/octet-stream
+         v
 [X-Content-Type-Options: nosniff] --> Browser won't sniff
          |
 [CSP: default-src 'none'] --> No script execution
 ```
 
-### Polyglot Attack Mitigations
+### Security Headers (Primary Defense)
 
-| Attack | Detection | Result |
+| Header | Value | Protection |
-|--------|-----------|--------|
+|--------|-------|------------|
-| PNG + HTML | PNG magic detected | image/png |
+| X-Content-Type-Options | nosniff | Prevents MIME sniffing |
-| GIF + JS | GIF magic detected | image/gif |
+| Content-Security-Policy | default-src 'none' | Blocks script execution |
-| PDF + ZIP | PDF magic detected | application/pdf |
+| X-Frame-Options | DENY | Prevents framing |
-| SVG + script | No XML magic | text/plain |
-| JPEG + PHP | JPEG magic detected | image/jpeg |
 
 ---