From a58848395c0cc94ff2dea5a5b3e33058691210c8 Mon Sep 17 00:00:00 2001 From: user Date: Thu, 19 Feb 2026 18:31:20 +0100 Subject: [PATCH] docs: rewrite all documentation for stealth connect and current state Update README, PROJECT, ROADMAP, TASKS, TODO, USAGE, CHEATSHEET, INSTALL, and DEBUG to reflect stealth connect, probation window, markov nick generation, local DNS resolution, and multi-IP failover. Co-Authored-By: Claude Opus 4.6 --- PROJECT.md | 43 ++++++++---- README.md | 26 ++++--- ROADMAP.md | 18 ++--- TASKS.md | 9 ++- TODO.md | 7 +- docs/CHEATSHEET.md | 104 +++++++++++++++++++-------- docs/DEBUG.md | 170 +++++++++++++++++++++++++++++---------------- docs/INSTALL.md | 44 ++++++++++-- docs/USAGE.md | 122 +++++++++++++++++++++++++++++--- 9 files changed, 402 insertions(+), 141 deletions(-) diff --git a/PROJECT.md b/PROJECT.md index dc88188..73ace48 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -2,36 +2,55 @@ ## Purpose -IRC bouncer that maintains persistent connections to IRC servers through a SOCKS5 proxy, allowing IRC clients to connect/disconnect while keeping the session alive and replaying missed messages. +IRC bouncer that maintains persistent connections to IRC servers through a +SOCKS5 proxy, allowing IRC clients to connect/disconnect while keeping the +session alive and replaying missed messages. ## Architecture ``` IRC Client(s) --> [bouncer:6667] --> Router --> [SOCKS5:1080] --> IRC Server(s) - | - Backlog - (SQLite) + | + Backlog + (SQLite) ``` +### Connection State Machine + +``` +DISCONNECTED -> CONNECTING -> REGISTERING -> PROBATION (15s) -> READY + | | | + '--------------+--------------' + (failure = backoff + retry) +``` + +- **CONNECTING**: SOCKS5 + optional TLS handshake +- **REGISTERING**: Random nick/user/realname sent to server +- **PROBATION**: 15s window to detect K-lines before exposing real identity +- **READY**: Configured nick set, channels joined, clients served + ### Components | Module | Responsibility | |--------|---------------| | `irc.py` | IRC protocol parser/formatter (RFC 2812 subset) | | `config.py` | TOML configuration loading and validation | -| `proxy.py` | SOCKS5 async connection wrapper | -| `network.py` | Persistent IRC server connection per network | +| `proxy.py` | SOCKS5 async connector with local DNS + multi-IP failover | +| `network.py` | Server connection state machine, stealth registration | | `server.py` | TCP listener accepting IRC client connections | -| `client.py` | Per-client session and IRC handshake | +| `client.py` | Per-client session, PASS/NICK/USER handshake | | `router.py` | Message routing between clients and networks | | `backlog.py` | SQLite message storage and replay | ### Key Decisions -- **asyncio**: Single-threaded async for all I/O -- **python-socks**: Async SOCKS5 proxy support -- **aiosqlite**: Non-blocking SQLite for backlog -- **No IRC library**: Manual protocol handling (IRC is simple line-based) +- **asyncio**: single-threaded async for all I/O +- **python-socks**: async SOCKS5 proxy support +- **aiosqlite**: non-blocking SQLite for backlog +- **No IRC library**: manual protocol handling (IRC is line-based) +- **Markov nicks**: English bigram frequencies for pronounceable random nicks +- **Local DNS**: resolve before SOCKS5 to handle proxies without DNS support +- **Multi-IP**: try all resolved addresses, skip unreachable exit IPs ## Dependencies @@ -43,4 +62,4 @@ IRC Client(s) --> [bouncer:6667] --> Router --> [SOCKS5:1080] --> IRC Server(s) ## Requirements - Python 3.10+ -- SOCKS5 proxy running on 127.0.0.1:1080 +- SOCKS5 proxy on 127.0.0.1:1080 diff --git a/README.md b/README.md index bbd8a31..0aee860 100644 --- a/README.md +++ b/README.md @@ -6,25 +6,24 @@ IRC bouncer with SOCKS5 proxy support and persistent message backlog. - Connect to multiple IRC networks simultaneously - All outbound connections routed through SOCKS5 proxy +- Stealth connect: registers with a random pronounceable nick and generic identity +- Probation window: waits 15s after registration to detect K-lines before revealing real nick - Persistent message backlog (SQLite) with replay on reconnect - Multiple clients can attach to the same network session - Password authentication - TLS support for IRC server connections - Automatic reconnection with exponential backoff -- Nick collision handling +- Local DNS resolution with multi-address failover ## Quick Start ```bash -# Clone and install cd ~/git/bouncer make dev -# Copy and edit config cp config/bouncer.example.toml config/bouncer.toml $EDITOR config/bouncer.toml -# Run bouncer -c config/bouncer.toml -v ``` @@ -38,9 +37,20 @@ PASS networkname:yourpassword Where `networkname` matches a `[networks.NAME]` section in your config. -## Configuration +## How It Works -See [config/bouncer.example.toml](config/bouncer.example.toml) for a full example. +``` +IRC Client(s) --> [bouncer:6667] --> Router --> [SOCKS5:1080] --> IRC Server(s) + | + Backlog + (SQLite) +``` + +1. Bouncer connects to IRC server via SOCKS5 with a random identity +2. Survives 15s probation (K-line detection) +3. Switches to your configured nick +4. Joins configured channels +5. Clients connect to bouncer, receive backlog replay and channel state ## Documentation @@ -59,7 +69,3 @@ make test # Run tests make lint # Run linter make fmt # Format code ``` - -## License - -MIT diff --git a/ROADMAP.md b/ROADMAP.md index a3b25d7..1d03f19 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -4,35 +4,37 @@ - [x] IRC protocol parser/formatter - [x] TOML configuration -- [x] SOCKS5 proxy connector +- [x] SOCKS5 proxy connector with local DNS + multi-IP failover - [x] Multi-network support - [x] Client authentication (password) - [x] Persistent backlog (SQLite) - [x] Backlog replay on reconnect -- [x] Automatic reconnection with backoff +- [x] Automatic reconnection with exponential backoff - [x] Nick collision handling - [x] TLS support +- [x] Stealth connect (random markov-generated identity) +- [x] Probation window (K-line detection before revealing nick) +- [x] Verified end-to-end on Libera.Chat via SOCKS5 ## v0.2.0 - [ ] Client-side TLS (accept TLS from clients) -- [ ] Per-network password support -- [ ] CTCP version/ping response -- [ ] Channel key support (JOIN #channel key) - [ ] SASL authentication to IRC servers -- [ ] Configurable backlog format (timestamps) +- [ ] CTCP VERSION/PING response +- [ ] Channel key support (JOIN #channel key) +- [ ] Configurable probation duration +- [ ] Configurable backlog timestamp format ## v0.3.0 -- [ ] Web status page - [ ] Hot config reload (SIGHUP) - [ ] Systemd service file - [ ] Per-client backlog tracking (multi-user) +- [ ] Web status page - [ ] DCC passthrough ## v1.0.0 - [ ] Stable API - [ ] Comprehensive test coverage -- [ ] Documentation complete - [ ] Packaged for PyPI diff --git a/TASKS.md b/TASKS.md index 02288c9..d3b4e40 100644 --- a/TASKS.md +++ b/TASKS.md @@ -5,9 +5,12 @@ - [x] P0: Core implementation (irc, config, proxy, network, client, server, router, backlog) - [x] P0: Unit tests (irc, config, backlog) - [x] P0: CLI and entry point -- [x] P0: Documentation -- [ ] P1: Integration testing with live IRC server -- [ ] P1: Verify SOCKS5 proxy connectivity end-to-end +- [x] P0: Stealth connect + probation state machine +- [x] P0: Markov bigram nick generator +- [x] P0: Local DNS resolution + multi-IP failover +- [x] P1: Integration testing with live IRC server (Libera.Chat) +- [x] P1: Verified SOCKS5 proxy connectivity end-to-end +- [x] P1: Documentation update ## Next diff --git a/TODO.md b/TODO.md index 266fd94..43115a1 100644 --- a/TODO.md +++ b/TODO.md @@ -7,6 +7,7 @@ - [ ] Channel key support - [ ] CTCP VERSION/PING responses - [ ] Hot config reload on SIGHUP +- [ ] Configurable probation duration - [ ] Web status dashboard - [ ] DCC passthrough @@ -22,8 +23,4 @@ - [ ] SOCKS5 proxy failure tests - [ ] Backlog replay edge cases - [ ] Concurrent client attach/detach - -## Documentation - -- [ ] Architecture diagram -- [ ] Sequence diagrams for connection flow +- [ ] Probation timeout / K-line detection tests diff --git a/docs/CHEATSHEET.md b/docs/CHEATSHEET.md index 115990c..75cb021 100644 --- a/docs/CHEATSHEET.md +++ b/docs/CHEATSHEET.md @@ -1,53 +1,101 @@ # Cheatsheet -## Commands +## Run ```bash -bouncer -c config/bouncer.toml # Start with config -bouncer -c config/bouncer.toml -v # Start with debug output -bouncer --version # Show version -bouncer --help # Show help +bouncer -c config/bouncer.toml # start +bouncer -c config/bouncer.toml -v # start (debug) +bouncer --version # version +bouncer --help # help ``` -## Development +## Develop ```bash -make dev # Install with dev deps -make test # Run pytest -make lint # Run ruff -make fmt # Format with black + ruff -make run # Run with default config -make clean # Remove .venv and build artifacts +make dev # install with dev deps into .venv +make test # pytest +make lint # ruff check +make fmt # black + ruff --fix +make run # run with config/bouncer.toml +make clean # rm .venv, build artifacts ``` -## Client Connection +## Client Auth ``` -PASS : # Authenticate + select network -PASS # Authenticate, use first network +PASS : # select network + authenticate +PASS # use first network ``` -## Config Structure +## Connection States + +``` +DISCONNECTED -> CONNECTING -> REGISTERING -> PROBATION (15s) -> READY +``` + +| State | What happens | +|-------|-------------| +| CONNECTING | TCP + SOCKS5 + TLS handshake | +| REGISTERING | Random nick/user/realname sent to server | +| PROBATION | 15s wait, watching for K-line | +| READY | Switch to configured nick, join channels | + +## Reconnect Backoff + +``` +5s -> 10s -> 30s -> 60s -> 120s -> 300s (cap) +``` + +## Config Skeleton ```toml -[bouncer] # Listener settings - bind / port / password - [bouncer.backlog] # Backlog settings +[bouncer] +bind / port / password + [bouncer.backlog] max_messages / replay_on_connect -[proxy] # SOCKS5 proxy - host / port +[proxy] +host / port -[networks.] # IRC server (repeatable) - host / port / tls - nick / user / realname - channels / autojoin / password +[networks.] # repeatable +host / port / tls +nick / channels / autojoin +password # optional, IRC server PASS ``` ## Files | Path | Purpose | |------|---------| -| `config/bouncer.toml` | Active configuration | -| `config/bouncer.db` | SQLite backlog database | -| `config/bouncer.example.toml` | Example config template | +| `config/bouncer.toml` | Active config (gitignored) | +| `config/bouncer.example.toml` | Example template | +| `config/bouncer.db` | SQLite backlog (auto-created) | + +## Backlog Queries + +```sql +-- recent messages +SELECT * FROM messages ORDER BY id DESC LIMIT 20; + +-- per-network counts +SELECT network, COUNT(*) FROM messages GROUP BY network; + +-- last seen state +SELECT * FROM client_state; +``` + +## Source Layout + +``` +src/bouncer/ + __main__.py # entry point, event loop + cli.py # argparse + config.py # TOML loader + irc.py # IRC message parse/format + proxy.py # SOCKS5 connector (local DNS, multi-IP) + network.py # server connection + state machine + client.py # client session handler + router.py # message routing + backlog trigger + server.py # TCP listener + backlog.py # SQLite store/replay/prune +``` diff --git a/docs/DEBUG.md b/docs/DEBUG.md index d1a246f..4a9bf34 100644 --- a/docs/DEBUG.md +++ b/docs/DEBUG.md @@ -6,65 +6,8 @@ bouncer -c config/bouncer.toml -v ``` -Debug logging shows: -- SOCKS5 proxy connection attempts -- IRC server registration -- Client connect/disconnect events -- Message routing -- Backlog replay counts - -## Common Issues - -### "config not found" - -Ensure the config path is correct: - -```bash -bouncer -c /full/path/to/bouncer.toml -``` - -### Connection refused (SOCKS5 proxy) - -Verify the proxy is running: - -```bash -ss -tlnp | grep 1080 -``` - -### Connection timeout to IRC server - -Check the SOCKS5 proxy can reach the IRC server: - -```bash -curl --socks5 127.0.0.1:1080 -v telnet://irc.libera.chat:6697 -``` - -### Nick already in use - -The bouncer appends `_` to the nick and retries. Check logs for: - -``` -WARNING bouncer.network [libera] nick in use, trying mynick_ -``` - -### TLS certificate errors - -If connecting to a server with a self-signed cert, this is currently not supported. All TLS connections use the system CA store. - -## Inspecting the Backlog Database - -```bash -sqlite3 config/bouncer.db - --- Recent messages -SELECT * FROM messages ORDER BY id DESC LIMIT 20; - --- Messages per network -SELECT network, COUNT(*) FROM messages GROUP BY network; - --- Client state -SELECT * FROM client_state; -``` +Shows: SOCKS5 connections, DNS resolution, IRC registration, state transitions, +K-line detection, nick changes, channel joins, client events, backlog replay. ## Log Format @@ -72,4 +15,111 @@ SELECT * FROM client_state; HH:MM:SS LEVEL module message ``` -Levels: `DEBUG`, `INFO`, `WARNING`, `ERROR` +## Connection States in Logs + +| Log message | Meaning | +|-------------|---------| +| `connecting to ...` | SOCKS5 + TLS handshake starting | +| `connected, registering as ` | Random identity sent to server | +| `registered as ` | Server accepted registration (001) | +| `probation started (15s)` | Watching for K-line | +| `server ERROR: ... (K-Lined)` | K-lined during probation | +| `probation passed, connection stable` | Safe to reveal identity | +| `switching nick: -> ` | Changing to configured nick | +| `nick changed: -> ` | Server confirmed nick change | +| `joined #channel` | Channel join successful | +| `reconnecting in Ns (attempt N)` | Backoff before retry | + +## Common Issues + +### K-lined on every attempt + +All SOCKS5 exit IPs may be banned on the target network. The bouncer will +keep retrying with exponential backoff, cycling through different exit IPs +(DNS round-robin). This is expected behavior -- it will eventually find a +clean IP if one exists. + +Check which IPs are being tried: + +```bash +bouncer -c config/bouncer.toml -v 2>&1 | grep 'trying\|K-Lined\|Banned' +``` + +### "config not found" + +```bash +bouncer -c /full/path/to/bouncer.toml +``` + +### SOCKS5 proxy not running + +```bash +ss -tlnp | grep 1080 +``` + +### SOCKS5 proxy can't reach server + +Test connectivity directly: + +```bash +curl --socks5 127.0.0.1:1080 -v telnet://irc.libera.chat:6697 +``` + +If remote DNS fails, test with local resolution: + +```bash +curl --socks5 127.0.0.1:1080 -v https://$(getent ahostsv4 irc.libera.chat | head -1 | awk '{print $1}'):6697 -k +``` + +### Nick already in use + +During probation (random nick): bouncer generates another random nick. +After probation (configured nick): bouncer appends `_` and retries. + +``` +WARNING bouncer.network [libera] nick in use, trying mynick_ +``` + +### TLS certificate errors + +All TLS connections use the system CA store. Self-signed certs are not +currently supported. + +### Connection drops after registration + +Typical for networks that ban proxy/VPN IPs. The bouncer handles this: +detects the K-line, waits with backoff, reconnects with fresh identity. + +## Inspecting the Backlog + +```bash +sqlite3 config/bouncer.db +``` + +```sql +-- recent messages +SELECT datetime(timestamp, 'unixepoch', 'localtime') as time, + network, sender, target, command, content +FROM messages ORDER BY id DESC LIMIT 20; + +-- message counts per network +SELECT network, COUNT(*) as msgs FROM messages GROUP BY network; + +-- last client disconnect per network +SELECT network, last_seen_id, + datetime(last_disconnect, 'unixepoch', 'localtime') as disconnected +FROM client_state; + +-- search messages +SELECT * FROM messages WHERE content LIKE '%keyword%' ORDER BY id DESC LIMIT 10; +``` + +## Watching Live Logs + +Run bouncer in foreground with verbose: + +```bash +bouncer -c config/bouncer.toml -v 2>&1 | grep -v aiosqlite +``` + +The `grep -v aiosqlite` filters out noisy SQLite debug lines. diff --git a/docs/INSTALL.md b/docs/INSTALL.md index ef203c4..c377aab 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -12,7 +12,8 @@ cd ~/git/bouncer make dev ``` -This creates `.venv/`, installs dependencies, and registers the `bouncer` command. +This creates `.venv/`, installs all dependencies, and registers the `bouncer` +command via editable install. ## Verify @@ -26,17 +27,46 @@ bouncer --version cp config/bouncer.example.toml config/bouncer.toml ``` -Edit `config/bouncer.toml` with your network details. At minimum, set: +Edit `config/bouncer.toml`. Required settings: -- `bouncer.password` -- client authentication password -- `networks..host` -- IRC server hostname -- `networks..nick` -- your IRC nickname -- `networks..channels` -- channels to auto-join +| Key | Description | +|-----|-------------| +| `bouncer.password` | Password for client authentication | +| `networks..host` | IRC server hostname | +| `networks..nick` | Your desired IRC nick | + +Optional but recommended: + +| Key | Default | Description | +|-----|---------|-------------| +| `networks..tls` | `false` | Enable TLS to IRC server | +| `networks..port` | `6667`/`6697` | Server port (auto-set by tls) | +| `networks..channels` | `[]` | Channels to auto-join | +| `networks..autojoin` | `true` | Join channels after probation | ## Symlink -The `make dev` editable install registers `bouncer` in `.venv/bin/`. To make it available system-wide: +To make `bouncer` available system-wide: ```bash ln -sf ~/git/bouncer/.venv/bin/bouncer ~/.local/bin/bouncer ``` + +Verify: + +```bash +which bouncer +``` + +## Dependencies + +Installed automatically by `make dev`: + +| Package | Purpose | +|---------|---------| +| `python-socks[asyncio]` | Async SOCKS5 proxy support | +| `aiosqlite` | Async SQLite for backlog | +| `ruff` | Linter (dev) | +| `black` | Formatter (dev) | +| `pytest` | Tests (dev) | +| `pytest-asyncio` | Async test support (dev) | diff --git a/docs/USAGE.md b/docs/USAGE.md index 73e53eb..a3d4661 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -12,6 +12,68 @@ bouncer -c config/bouncer.toml -v | `-v, --verbose` | Debug logging | | `--version` | Show version | +## Connection Lifecycle + +The bouncer goes through several states when connecting to an IRC server: + +``` +DISCONNECTED -> CONNECTING -> REGISTERING -> PROBATION -> READY + | | | + `--------------+--------------' + (failure = reconnect) +``` + +### 1. Stealth Registration + +On connect, the bouncer registers with a **random identity**: + +- **Nick**: pronounceable markov-generated word (e.g., `heliagu`, `crewo`, `midon`) +- **User/Ident**: random pronounceable word +- **Realname**: random capitalized word + +No fixed prefix or pattern -- each attempt looks like a different person. + +### 2. Probation (15 seconds) + +After registration succeeds (001 RPL_WELCOME), the bouncer enters a 15-second +probation window. During this time it watches for: + +- `ERROR` messages (K-line, ban) +- Server closing the connection + +If the connection drops during probation, the bouncer reconnects with a fresh +random identity and tries again. + +### 3. Ready + +Once probation passes without incident: + +1. Bouncer switches to your configured nick (`NICK mynick`) +2. Joins configured channels (if `autojoin = true`) +3. Begins relaying messages to/from connected clients + +### 4. Reconnection + +On any disconnection, the bouncer reconnects with exponential backoff: + +| Attempt | Delay | +|---------|-------| +| 1 | 5s | +| 2 | 10s | +| 3 | 30s | +| 4 | 60s | +| 5 | 120s | +| 6+ | 300s | + +Each reconnection uses a fresh random identity. + +## DNS Resolution + +Hostnames are resolved locally before being passed to the SOCKS5 proxy. If a +hostname resolves to multiple IPs, the bouncer tries each one until a connection +succeeds. This handles proxies that don't support remote DNS and avoids IPs +that are unreachable through the proxy. + ## Connecting with an IRC Client Configure your IRC client to connect to the bouncer: @@ -31,7 +93,8 @@ PASS : - `network` -- matches a `[networks.NAME]` section in config - `password` -- the `bouncer.password` value from config -If you omit the network prefix (`PASS yourpassword`), the first configured network is used. +If you omit the network prefix (`PASS yourpassword`), the first configured +network is used. ### Client Examples @@ -47,24 +110,38 @@ If you omit the network prefix (`PASS yourpassword`), the first configured netwo ``` **hexchat:** + Set server password to `libera:mypassword` in the network settings. ## Multiple Networks -Define multiple `[networks.*]` sections in the config. Connect with different passwords to access each: +Define multiple `[networks.*]` sections in the config. Each gets its own +persistent server connection through the SOCKS5 proxy. + +Connect your client with the appropriate network prefix: ``` -PASS libera:mypassword # connects to libera -PASS oftc:mypassword # connects to oftc +PASS libera:mypassword # connects to [networks.libera] +PASS oftc:mypassword # connects to [networks.oftc] ``` -Multiple clients can attach to the same network simultaneously. +Multiple clients can attach to the same network simultaneously. All receive +the same messages in real time. + +## What Clients Receive on Connect + +When a client authenticates and attaches to a network: + +1. **Backlog replay** -- missed messages since last disconnect +2. **Synthetic welcome** -- 001-004 numeric replies from the bouncer +3. **Channel state** -- TOPIC and NAMES for each joined channel ## Backlog -Messages are stored in `bouncer.db` (SQLite) next to the config file. When you reconnect, missed messages are automatically replayed. +Messages are stored in `bouncer.db` (SQLite) next to the config file. When +you reconnect, missed messages are automatically replayed. -Configure backlog in `bouncer.toml`: +Configure in `bouncer.toml`: ```toml [bouncer.backlog] @@ -72,6 +149,35 @@ max_messages = 10000 # per network, 0 = unlimited replay_on_connect = true # set false to disable replay ``` +Stored commands: `PRIVMSG`, `NOTICE`, `TOPIC`, `KICK`, `MODE`. + +## Configuration Reference + +```toml +[bouncer] +bind = "127.0.0.1" # listen address +port = 6667 # listen port +password = "changeme" # client authentication password + +[bouncer.backlog] +max_messages = 10000 # per network, 0 = unlimited +replay_on_connect = true # replay missed messages on client connect + +[proxy] +host = "127.0.0.1" # SOCKS5 proxy address +port = 1080 # SOCKS5 proxy port + +[networks.libera] +host = "irc.libera.chat" # IRC server hostname +port = 6697 # server port (default: 6697 if tls, 6667 otherwise) +tls = true # use TLS for server connection +nick = "mynick" # desired IRC nick (set after probation) +channels = ["#test"] # channels to join (after probation) +autojoin = true # auto-join channels on ready (default: true) +password = "" # IRC server password (optional, for PASS command) +``` + ## Stopping -Press `Ctrl+C` or send `SIGTERM`. The bouncer shuts down gracefully. +Press `Ctrl+C` or send `SIGTERM`. The bouncer shuts down gracefully, closing +all network connections and the backlog database.