derp/TASKS.md

derp - Tasks

Current Sprint -- Discovery Playlists (2026-02-23)

Pri	Status	Task
P0	[x]	!similar default: discover + resolve + play (playlist mode)
P0	[x]	!similar list subcommand for display-only (old default)
P0	[x]	_search_queries() normalizes Last.fm/MB results to search strings
P0	[x]	_resolve_playlist() parallel yt-dlp resolution via ThreadPoolExecutor
P1	[x]	Playback transition: fade out, clear queue, load playlist, fade in
P1	[x]	Fallback to display when music plugin not loaded
P1	[x]	Tests: 11 new cases (81 total in test_lastfm.py, 1949 suite total)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, TASKS.md)

Previous Sprint -- Enhanced Help with FlaskPaste (2026-02-23)

Pri	Status	Task
P0	[x]	!help <cmd> pastes docstring detail via FlaskPaste, appends URL
P0	[x]	!help <plugin> pastes all plugin command details
P0	[x]	!help (no args) pastes full reference grouped by plugin
P1	[x]	3-level hierarchy: plugin (col 0), command (indent 4), docstring (indent 8)
P1	[x]	Graceful fallback when FlaskPaste not loaded or paste fails
P1	[x]	Helper functions: _build_cmd_detail(indent=), _paste
P1	[x]	Tests: 7 new cases in test_core.py (11 total)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, TASKS.md)

Previous Sprint -- MusicBrainz Fallback (2026-02-23)

Pri	Status	Task
P0	[x]	!similar MusicBrainz fallback when no Last.fm API key or empty results
P0	[x]	!tags MusicBrainz fallback when no Last.fm API key or empty results
P1	[x]	!similar play works through MB fallback path
P1	[x]	Tests: MB fallback for both commands (6 new cases, 70 total in test_lastfm.py)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, ROADMAP.md, TODO.md, PROJECT.md)

Previous Sprint -- Voice + Music UX (2026-02-22)

Pri	Status	Task
P0	[x]	Acknowledge tone (880Hz/1320Hz chime) before TTS playback
P0	[x]	Duck-before-TTS: 1.5s delay for music to lower before audio starts
P0	[x]	Instant packet-based ducking via pymumble sound callback (~20ms)
P0	[x]	Duck floor raised to 2% (keep music audible during voice)
P0	[x]	Strip leading punctuation from voice trigger remainder
P0	[x]	Fix greeting tests: move greet TTS to voice plugin on_connected
P0	[x]	Whisper initial_prompt bias for trigger word recognition
P1	[x]	Queue display improvements (!queue shows elapsed/duration, totals)
P1	[x]	Playlist save/load/list/del (!playlist save <name>, etc.)
P2	[ ]	Per-channel voice settings (different voice per Mumble channel)

Previous Sprint -- Performance: HTTP + Parsing (2026-02-22)

Pri	Status	Task
P0	[x]	Rewrite _extract_videos as iterative stack-based (51K recursive calls from 4 invocations)
P0	[x]	plugins/searx.py -- route through derp.http.urlopen(proxy=False)
P1	[x]	Connection pool: preload_content=True + _PooledResponse wrapper for connection reuse
P1	[x]	Pool tuning: num_pools=30, maxsize=8 (was 20/4)
P2	[x]	Audit remaining plugins for unnecessary proxy routing

Previous Sprint -- Music Discovery via Last.fm (2026-02-22)

Pri	Status	Task
P0	[x]	plugins/lastfm.py -- Last.fm API client (artist.getSimilar, artist.getTopTags, track.getSimilar)
P0	[x]	!similar command -- show similar artists for current or named track/artist
P0	[x]	!similar play -- queue a similar track via YouTube search
P1	[x]	!tags command -- show genre/style tags for current or named track
P1	[x]	Config: [lastfm] api_key or LASTFM_API_KEY env var
P2	[x]	Tests: test_lastfm.py (50 cases: API helpers, metadata, commands)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md)

Previous Sprint -- v2.3.0 Mumble Voice + Multi-Bot (2026-02-22)

Pri	Status	Task
P0	[x]	src/derp/mumble.py -- rewrite to pymumble transport (voice + text)
P0	[x]	plugins/music.py -- play/stop/skip/queue/np/volume/seek/resume
P0	[x]	plugins/voice.py -- STT (Whisper) + TTS (Piper), voice profiles
P0	[x]	Container patches for pymumble ssl + opuslib musl
P0	[x]	Multi-bot Mumble ([[mumble.extra]]), per-bot plugin filtering
P0	[x]	Rubberband pitch-shifting via CLI (Containerfile + FX chain split)
P0	[x]	Bot audio ignored in sound callback (no self-ducking/STT of bots)
P0	[x]	Self-mute support (mute on join, unmute for audio, re-mute after)
P1	[x]	plugins/alias.py -- command aliases (add/del/list)
P1	[x]	Container management tools (tools/build,start,stop,restart,nuke,logs,status)
P1	[x]	Tests: test_mumble.py, test_music.py, test_alias.py, test_core.py
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, ROADMAP.md)

Previous Sprint -- v2.3.0 Mumble Music Playback (2026-02-21)

Pri	Status	Task
P0	[x]	src/derp/mumble.py -- rewrite to pymumble transport (voice + text)
P0	[x]	plugins/music.py -- play/stop/skip/queue/np/volume commands
P0	[x]	Container patches for pymumble ssl + opuslib musl
P1	[x]	Tests: test_mumble.py (62 cases), test_music.py (28 cases)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md)

Previous Sprint -- v2.2.0 Configurable Proxy (2026-02-21)

Pri	Status	Task
P0	[x]	src/derp/http.py -- proxy parameter on all public functions
P0	[x]	src/derp/config.py -- proxy defaults per adapter section
P0	[x]	src/derp/irc.py -- optional SOCKS5 for IRC connections
P0	[x]	src/derp/telegram.py -- pass proxy config to HTTP calls
P0	[x]	src/derp/teams.py -- pass proxy config to HTTP calls
P0	[x]	src/derp/mumble.py -- pass proxy config to TCP calls
P1	[x]	Tests: proxy toggle paths (24 new cases, 1494 total)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, API.md)

Previous Sprint -- v2.2.0 Mumble Adapter (2026-02-21)

Pri	Status	Task
P0	[x]	src/derp/mumble.py -- MumbleBot, MumbleMessage, protobuf codec
P0	[x]	TCP/TLS connection through SOCKS5 proxy
P0	[x]	Minimal protobuf encoder/decoder (no external protobuf dep)
P0	[x]	Mumble protocol: Version, Authenticate, Ping, TextMessage
P0	[x]	Channel/user state tracking from ChannelState/UserState messages
P0	[x]	src/derp/config.py -- [mumble] defaults
P0	[x]	src/derp/cli.py -- conditionally start MumbleBot
P1	[x]	Tests: test_mumble.py (93 cases, 1470 total)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, API.md, README.md, ROADMAP.md)

Previous Sprint -- v2.1.0 Telegram Integration (2026-02-21)

Pri	Status	Task
P0	[x]	Fix src/derp/teams.py -- route send() through SOCKS5 proxy
P0	[x]	src/derp/telegram.py -- TelegramBot, TelegramMessage, long-polling
P0	[x]	src/derp/config.py -- [telegram] defaults
P0	[x]	src/derp/cli.py -- conditionally start TelegramBot
P0	[x]	All Telegram HTTP through SOCKS5 proxy (derp.http.urlopen)
P0	[x]	Permission tiers via user IDs (exact match)
P0	[x]	@botusername suffix stripping, message splitting (4096 chars)
P1	[x]	Tests: test_telegram.py (75 cases)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, API.md, README.md, ROADMAP.md)

Previous Sprint -- v2.1.0 Teams Integration (2026-02-21)

Pri	Status	Task
P0	[x]	src/derp/teams.py -- TeamsBot, TeamsMessage, HTTP handler
P0	[x]	src/derp/config.py -- [teams] defaults
P0	[x]	src/derp/cli.py -- conditionally start TeamsBot alongside IRC bots
P0	[x]	HMAC-SHA256 signature validation (base64 key, Authorization: HMAC header)
P0	[x]	Permission tiers via AAD object IDs (exact match, not fnmatch)
P0	[x]	IRC no-ops: join, part, kick, mode, set_topic (debug log)
P1	[x]	Tests: test_teams.py (74 cases, 1302 total)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, API.md, README.md, ROADMAP.md)

Previous Sprint -- v2.0.0 Stable API (2026-02-21)

Pri	Status	Task
P0	[x]	Version bump to 2.0.0 (__init__.py, pyproject.toml)
P0	[x]	docs/API.md -- plugin API reference with semver policy
P2	[x]	Documentation update (README.md, ROADMAP.md, TODO.md, TASKS.md)

Previous Sprint -- v2.0.0 Multi-Server (2026-02-21)

Pri	Status	Task
P0	[x]	build_server_configs() in src/derp/config.py (legacy + multi layout)
P0	[x]	Bot.__init__ signature: name, _pstate, per-server state DB path
P0	[x]	cli.py multi-bot loop: concurrent asyncio.gather, shared registry
P0	[x]	9 stateful plugins migrated to _ps(bot) pattern (rss, yt, twitch, alert, cron, pastemoni, remind, webhook, urltitle)
P0	[x]	core.py -- !version shows bot.name
P1	[x]	All affected tests updated (Bot signature, FakeBot._pstate, state access)
P1	[x]	New tests: TestServerName (6), TestBuildServerConfigs (10)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, ROADMAP.md, TODO.md)

Previous Sprint -- v2.0.0 ACL + Webhook (2026-02-21)

Pri	Status	Task
P0	[x]	Granular ACL tiers in src/derp/plugin.py (TIERS, Handler.tier, decorator)
P0	[x]	ACL dispatch in src/derp/bot.py (_get_tier, _operators, _trusted)
P0	[x]	Config defaults: operators, trusted, webhook section
P0	[x]	plugins/core.py -- whoami/admins tier display
P0	[x]	plugins/webhook.py -- HTTP webhook listener (HMAC, JSON, POST)
P1	[x]	Tests: test_acl.py (32 cases), test_webhook.py (22 cases)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, ROADMAP.md, TODO.md)

Previous Sprint -- v2.0.0 Tier 2 (2026-02-21)

Pri	Status	Task
P0	[x]	Bot.shorten_url() method in src/derp/bot.py
P0	[x]	URL shortening in rss.py, youtube.py, pastemoni.py announcements
P0	[x]	plugins/cron.py -- scheduled command execution (add/del/list)
P0	[x]	.gitea/workflows/ci.yml -- Gitea Actions CI pipeline
P1	[x]	Tests: test_flaskpaste.py (9 cases), test_cron.py (~38 cases)
P1	[x]	FakeBot shorten_url in test_rss, test_youtube, test_pastemoni
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, ROADMAP.md, TODO.md)

Previous Sprint -- v2.0.0 Quick Wins (2026-02-21)

Pri	Status	Task
P0	[x]	!paste command in plugins/flaskpaste.py
P0	[x]	Unit tests: test_encode.py (18 cases)
P0	[x]	Unit tests: test_hash.py (15 cases)
P0	[x]	Unit tests: test_defang.py (18 cases)
P0	[x]	Unit tests: test_cidr.py (14 cases)
P0	[x]	Unit tests: test_dns_plugin.py (18 cases)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md, ROADMAP.md, TODO.md)

Previous Sprint -- v1.3.0 Tier 2 Plugins (2026-02-20)

Pri	Status	Task
P0	[x]	Canary token generator (plugins/canary.py) -- gen/list/info/del
P0	[x]	TCP ping (plugins/tcping.py) -- latency probe via SOCKS5
P0	[x]	Wayback archive (plugins/archive.py) -- Save Page Now via SOCKS5
P0	[x]	Bulk DNS resolve (plugins/resolve.py) -- concurrent TCP DNS via SOCKS5
P1	[x]	Tests for all 4 plugins
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md)

Previous Sprint -- v1.2.9 InternetDB Plugin (2026-02-19)

Pri	Status	Task
P0	[x]	Shodan InternetDB plugin (plugins/internetdb.py) -- free, no API key
P0	[x]	Fetch via SOCKS5 proxy (derp.http.urlopen)
P1	[x]	Compact formatting: hostnames, ports, CPEs, tags, CVEs with truncation
P1	[x]	Input validation: IPv4/IPv6, private/loopback rejection
P2	[x]	Tests: fetch, format, command handler (21 cases, 927 total)
P2	[x]	Documentation update (USAGE.md, CHEATSHEET.md)

Previous Sprint -- v1.2.8 ASN Backend Replacement (2026-02-19)

Pri	Status	Task
P0	[x]	Replace MaxMind ASN with iptoasn.com TSV backend (no license key)
P0	[x]	Bisect-based lookup in plugins/asn.py (pure stdlib)
P1	[x]	update_asn() in scripts/update-data.sh (SOCKS5 download)
P2	[x]	Tests: load, lookup, command handler (30 cases, 906 total)
P2	[x]	Documentation update (USAGE.md data directory layout)

Previous Sprint -- v1.2.7 Subscription Plugin Enrichment (2026-02-19)

Pri	Status	Task
P0	[x]	Twitch: viewer count in live announcements (`
P0	[x]	YouTube: views, likes, published date in announcements (`
P0	[x]	RSS: published date in announcements (`
P1	[x]	Twitch check/list show viewer count
P1	[x]	RSS _parse_date helper (ISO + RFC 2822)
P2	[x]	Tests: twitch/youtube/rss enrichment (263 sub-plugin tests, 868 total)
P2	[x]	Documentation update (USAGE.md announcement formats)

Previous Sprint -- v1.2.6 Alert Backend Metadata Enrichment (2026-02-18)

Pri	Status	Task
P0	[x]	_compact_num helper (1k/1.2M formatting)
P0	[x]	DB migration: extra column in results table
P0	[x]	Backend metadata: 15 backends populate extra field
P1	[x]	Move engagement metrics from titles to extra (HN, GH, GL, SE, DH, HF, KK)
P1	[x]	Display: announcements, history, info show `
P2	[x]	Tests: TestCompactNum, extra in poll/history/info (91 total)
P2	[x]	Documentation update (USAGE.md metadata table)

Previous Sprint -- v1.2.5 Paste Site Keyword Monitor (2026-02-18)

Pri	Status	Task
P0	[x]	Pastemoni plugin (plugins/pastemoni.py)
P0	[x]	Pastebin archive scraping + raw content matching
P0	[x]	GitHub Gists API keyword filtering
P1	[x]	Polling/subscription architecture (rss.py pattern)
P1	[x]	State persistence + restore on connect
P1	[x]	Command handler: add/del/list/check
P2	[x]	Tests for pastemoni (15 test classes, ~45 cases)
P2	[x]	Documentation update (USAGE.md)

Previous Sprint -- v1.2.4 URL Title Preview (2026-02-17)

Pri	Status	Task
P0	[x]	URL title preview plugin (plugins/urltitle.py)
P0	[x]	HEAD-then-GET fetch via SOCKS5 connection pool
P1	[x]	_TitleParser: og:title/description + <title> fallback
P1	[x]	URL extraction with !-suppression and balanced parens
P1	[x]	Dedup/cooldown (5 min, 500 entry cache)
P1	[x]	Skip non-HTML, binary extensions, FlaskPaste host
P2	[x]	Tests for urltitle (11 test classes, ~40 cases)
P2	[x]	Documentation update (USAGE.md)

Previous Sprint -- v1.2.3 Paste Overflow (2026-02-17)

Pri	Status	Task
P0	[x]	Bot.long_reply() method with FlaskPaste overflow
P0	[x]	Configurable paste_threshold (default: 4)
P1	[x]	Refactor alert history to use long_reply()
P1	[x]	Refactor exploitdb search/cve to use long_reply()
P1	[x]	Refactor subdomain, crtsh, abuseipdb, dork to use long_reply()
P2	[x]	Tests for paste overflow (10 cases)

Previous Sprint -- v1.2.2 Connection Pooling + Batch OG (2026-02-17)

Pri	Status	Task
P0	[x]	Batch _fetch_og calls via ThreadPoolExecutor (alert.py)
P0	[x]	Connection pooling via urllib3[socks] SOCKSProxyManager (http.py)
P1	[x]	Cache FlaskPaste _ssl_context() at module level
P1	[x]	Backward-compat urllib.error.HTTPError for 4xx/5xx in pooled path
P1	[x]	Legacy opener fallback for context= callers (username.py)
P2	[x]	Containerfile uses requirements.txt for deps

Previous Sprint -- v1.2.1 Performance + Polish (2026-02-17)

Pri	Status	Task
P1	[x]	Cache default HTTP opener at module level
P1	[x]	--tracemalloc CLI flag for memory profiling
P1	[x]	Background seeding on !alert add (instant reply)
P1	[x]	Per-backend error tracking with exponential backoff
P1	[x]	Concurrent fetches for multi-instance backends (PeerTube, Mastodon, Lemmy, SearXNG)
P1	[x]	retries parameter for derp.http.urlopen
P2	[x]	Full alert titles (ACTION metadata + PRIVMSG content)
P2	[x]	Remove title truncation from backend builders

Completed

Date	Task
2026-02-23	!similar discovery playlists (parallel resolve, fade transition, list subcommand)
2026-02-23	Enhanced !help with FlaskPaste detail pages (docstrings, grouped reference)
2026-02-23	MusicBrainz fallback for !similar and !tags (no Last.fm key required)
2026-02-22	v2.3.0 (voice profiles, rubberband FX, multi-bot, self-mute, container tools)
2026-02-21	v2.3.0 (pymumble rewrite, music playback, fades, seek, kept library)
2026-02-17	v1.2.3 (paste overflow with FlaskPaste integration)
2026-02-17	v1.2.1 (HTTP opener cache, alert perf, concurrent multi-instance, tracemalloc)
2026-02-16	v1.2.0 (subscriptions, alerts, proxy, reminders)
2026-02-15	Calendar-based reminders (at/yearly) with persistence
2026-02-15	v1.1.0 (channel filter, JSON logging, dork, wayback, tests)
2026-02-15	v1.0.0 (IRCv3, chanmgmt, state persistence)
2026-02-15	Wave 4 (opslog, note, subdomain, headers, exploitdb, payload)
2026-02-15	Wave 3 plugins (geoip, asn, torcheck, iprep, cve) + update script
2026-02-15	Admin/owner permission system (hostmask + IRCOP)
2026-02-15	SASL PLAIN, rate limiting, CTCP responses
2026-02-15	Wave 2 plugins (whois, portcheck, httpcheck, tlscheck, blacklist, rand, timer)
2026-02-15	CLI --cprofile flag
2026-02-15	Wave 1 plugins (dns, encode, hash, defang, revshell, cidr)
2026-02-15	Hot-reload, shorthand, plugin help
2026-02-15	Container deployment (Containerfile, compose, Makefile targets)
2026-02-15	crt.sh CT lookup plugin
2026-02-15	TLS verify option for self-signed certs
2026-02-15	Initial implementation (IRC, plugins, config, CLI)