Music:
- #random URL fragment shuffles playlist tracks before enqueuing
- Lazy playlist resolution: first 10 tracks resolve immediately,
remaining are fetched in a background task
- !kept repair re-downloads kept tracks with missing local files
- !kept shows [MISSING] marker for tracks without local files
- TTS ducking: music ducks when merlin speaks via voice peer,
smooth restore after TTS finishes
Performance (from profiling):
- Connection pool: preload_content=True for SOCKS connection reuse
- Pool tuning: 30 pools / 8 connections (up from 20/4)
- _PooledResponse wrapper for stdlib-compatible read interface
- Iterative _extract_videos (replace 51K-deep recursion with stack)
- proxy=False for local SearXNG
Voice + multi-bot:
- Per-bot voice config lookup ([<username>.voice] in TOML)
- Mute detection: skip duck silence when all users muted
- Autoplay shuffle deck (no repeats until full cycle)
- Seek clamp to track duration (prevent seek-past-end stall)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Connect to multiple IRC servers concurrently from a single config file.
Plugins are loaded once and shared; per-server state is isolated via
separate SQLite databases and per-bot runtime state (bot._pstate).
- Add build_server_configs() for [servers.*] config layout
- Bot.__init__ gains name parameter, _pstate dict for plugin isolation
- cli.py runs multiple bots via asyncio.gather
- 9 stateful plugins migrated from module-level dicts to _ps(bot) pattern
- Backward compatible: legacy [server] config works unchanged
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Alert backends now populate structured `extra` field with engagement
metrics (views, stars, votes, etc.) instead of embedding them in titles.
Subscription plugins show richer announcements: Twitch viewer counts,
YouTube views/likes/dates, RSS published dates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Bot.long_reply() that sends lines directly when under threshold,
or creates a FlaskPaste paste with preview + link when over. Refactor
abuseipdb, alert history, crtsh, dork, exploitdb, and subdomain
plugins to use long_reply(). Configurable paste_threshold (default: 4).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-request SOCKS5+TLS handshakes with urllib3 SOCKSProxyManager
connection pool (20 pools, 4 conns/host). Batch _fetch_og calls via
ThreadPoolExecutor to parallelize OG tag enrichment in alert polling.
Cache flaskpaste SSL context at module level.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mastodon, Bluesky, GitHub, GitLab, npm, PyPI, and arXiv backends
no longer truncate content/descriptions in titles. Full text is
shown on the PRIVMSG line; only !alert history keeps truncation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ACTION carries the tag/date/URL, PRIVMSG carries the uncropped title.
Removes _truncate on alert output for better readability.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add _fetch_many() helper using ThreadPoolExecutor to query instances
in parallel. Refactors PeerTube, Mastodon, Lemmy, and SearXNG from
sequential to concurrent fetches. Also adds retries parameter to
derp.http.urlopen; multi-instance backends use retries=1 since
instance redundancy already provides resilience.
Worst-case wall time per backend drops from N*timeout to 1*timeout.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per-backend error counts with exponential backoff: after 5 consecutive
failures a backend is skipped every 2^(n-5) cycles (capped at 32).
Working backends are no longer penalized by one flaky backend doubling
the entire poll interval.
Migrates last_error (string) to last_errors (dict per backend).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reply immediately with empty seen, run a silent _poll_once in a
background task to populate seen IDs, then start the poller.
Eliminates the 30-120s blocking wait from 27 sequential backend queries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Store shortened URLs in the results DB at poll time alongside the
original URL. History output uses the stored short URL directly,
only regenerating (and persisting) when no short URL exists yet.
Original URL always preserved for re-shortening if needed.
- PoW-authenticated paste creation and URL shortening via FlaskPaste
- !paste <text> creates a paste, !shorten <url> shortens a URL
- Module-level shorten_url/create_paste helpers for cross-plugin use
- Alert plugin auto-shortens URLs in announcements and history output
- Custom TLS CA cert support via secrets/flaskpaste/derp.crt
- No SOCKS proxy -- direct urllib.request to FlaskPaste instance
Add Wikipedia, Stack Exchange, GitLab, npm, PyPI, Docker Hub,
arXiv, Lobsters, DEV.to, Medium, and Hugging Face backends to
the alert plugin (16 -> 27 total). Fix PyPI backend to use RSS
updates feed (web search now requires JS challenge). Fix DEV.to
to use public articles API (feed_content endpoint returns empty).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each alert result gets a deterministic 8-char base36 ID derived from
backend:item_id. IDs appear in announcements and history, and can be
looked up with !alert info <id> for full details. Existing rows are
backfilled on startup.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hacker News (hn) uses Algolia search_by_date API for stories,
appends point count to title, falls back to HN discussion URL
when no external link. GitHub (gh) searches repositories sorted
by recently updated, shows star count and truncated description.
Both routed through SOCKS5 proxy via _urlopen.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bluesky (bs) searches public post API, constructs bsky.app URLs
from at:// URIs. Lemmy (ly) queries 4 instances (lemmy.ml,
lemmy.world, programming.dev, infosec.pub) with cross-instance
dedup. Odysee (od) uses LBRY JSON-RPC claim_search for video,
audio, and documents with lbry:// to odysee.com URL conversion.
Archive.org (ia) searches via advanced search API sorted by date.
All routed through SOCKS5 proxy via _urlopen.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Kick (kk) searches channels and livestreams via public search API.
Dailymotion (dm) queries video API sorted by recent. PeerTube (pt)
searches across 4 federated instances with per-instance timeout.
All routed through SOCKS5 proxy via _urlopen.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DuckDuckGo (dg) searches via HTML lite endpoint with HTMLParser,
resolves DDG redirect URLs to actual targets. Google News (gn)
queries public RSS feed, parses RFC 822 dates. Both routed through
SOCKS5 proxy via _urlopen.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Query general, news, videos, and social media categories
separately with time_range=day. Dedup results by URL across
categories to avoid announcing the same item twice.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Search Reddit posts (rd) via JSON API and Mastodon hashtag
timelines (ft) across 4 fediverse instances. Both public,
no auth required.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Matched results were announced then discarded. Add a dedicated SQLite
database (data/alert_history.db) to store every announced result with
channel, alert name, backend, title, URL, date, and timestamp. Add
!alert history <name> [n] subcommand to query recent results.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
YouTube InnerTube is a public API -- no need to route through SOCKS5
proxy, which was causing SSL EOF errors on every poll. Switch to
direct urllib.request.urlopen.
Remove _MAX_ANNOUNCE cap; all matched results are now announced
individually instead of truncating with "... and N more".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SearXNG can return "publishedDate": null, which bypasses the default
value in dict.get() and passes None to _parse_date / re.search.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract dates from multiple sources:
- SearXNG: publishedDate field from search results
- YouTube: publishedTimeText from InnerTube response
- OG fallback: article:published_time, og:updated_time, date,
dc.date, dcterms.date, sailthru.date meta tags
Date is shown as (YYYY-MM-DD) or relative time after the tag prefix.
OG tags are fetched for date even when title/URL already matched.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a search result's title/URL doesn't contain the keyword, fetch
the page's first 64 KB and parse og:title and og:description meta
tags. If the keyword appears there, the result is announced. Prefers
og:title as display title when it's richer than the search result
title.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Search backends return loosely relevant results. Now only announces
items where the keyword actually appears in the title or URL
(case-insensitive). All results are still marked as seen to prevent
re-checking.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add exponential-backoff retry (3 attempts) for transient SSL,
connection, timeout, and OS errors to all three proxy functions:
urlopen, create_connection, open_connection. Remove per-plugin
retry from alert.py since transport layer now handles it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add retry loop (3 attempts, exponential backoff) for SSLError,
ConnectionError, TimeoutError, and OSError in alert poll cycle.
Non-transient errors fail immediately. Also fixes searx test
mocks to match direct urlopen usage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch to searx.mymx.me (public domain) for SearXNG queries
without routing through SOCKS5 proxy. Internal address is not
reachable from the container network.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SearXNG instance at 192.168.122.119 is reachable via grokbox
static route -- no need to tunnel through SOCKS5. Reverts searx
and alert plugins to stdlib urlopen for SearXNG queries. YouTube
and Twitch in alert.py still use the proxy. Also removes cprofile
flag from docker-compose command.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both plugins called urllib.request.urlopen directly, bypassing the
proxy. Switch to derp.http.urlopen and update the SearXNG endpoint
to the public domain (searx.mymx.me). Update test mocks to match.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add PySocks dependency and shared src/derp/http.py module providing
proxy-aware urlopen() and build_opener() that route through
socks5h://127.0.0.1:1080. Subclassed SocksiPyHandler passes SSL
context through to HTTPS connections.
Swapped 14 external-facing plugins to use the proxied helpers.
Local-only traffic (SearXNG, raw DNS/TLS sockets) stays direct.
Updated test mocks in test_twitch and test_alert accordingly.
Add standalone !searx command for on-demand SearXNG search (top 3 results).
Add SearX as a third backend (sx) to the alert plugin for keyword monitoring.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Search keywords across YouTube (InnerTube) and Twitch (GQL)
simultaneously, announcing new results per channel. Supports
add/del/list/check subcommands with per-platform seen lists.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
