YouTube InnerTube is a public API -- no need to route through SOCKS5
proxy, which was causing SSL EOF errors on every poll. Switch to
direct urllib.request.urlopen.
Remove _MAX_ANNOUNCE cap; all matched results are now announced
individually instead of truncating with "... and N more".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SearXNG can return "publishedDate": null, which bypasses the default
value in dict.get() and passes None to _parse_date / re.search.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
journald was dropping early startup logs. k8s-file writes directly
to disk, captures from process start, and is lighter on the Pi.
Capped at 10 MB with automatic rotation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract dates from multiple sources:
- SearXNG: publishedDate field from search results
- YouTube: publishedTimeText from InnerTube response
- OG fallback: article:published_time, og:updated_time, date,
dc.date, dcterms.date, sailthru.date meta tags
Date is shown as (YYYY-MM-DD) or relative time after the tag prefix.
OG tags are fetched for date even when title/URL already matched.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a search result's title/URL doesn't contain the keyword, fetch
the page's first 64 KB and parse og:title and og:description meta
tags. If the keyword appears there, the result is announced. Prefers
og:title as display title when it's richer than the search result
title.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Search backends return loosely relevant results. Now only announces
items where the keyword actually appears in the title or URL
(case-insensitive). All results are still marked as seen to prevent
re-checking.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add exponential-backoff retry (3 attempts) for transient SSL,
connection, timeout, and OS errors to all three proxy functions:
urlopen, create_connection, open_connection. Remove per-plugin
retry from alert.py since transport layer now handles it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add retry loop (3 attempts, exponential backoff) for SSLError,
ConnectionError, TimeoutError, and OSError in alert poll cycle.
Non-transient errors fail immediately. Also fixes searx test
mocks to match direct urlopen usage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch to searx.mymx.me (public domain) for SearXNG queries
without routing through SOCKS5 proxy. Internal address is not
reachable from the container network.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SearXNG instance at 192.168.122.119 is reachable via grokbox
static route -- no need to tunnel through SOCKS5. Reverts searx
and alert plugins to stdlib urlopen for SearXNG queries. YouTube
and Twitch in alert.py still use the proxy. Also removes cprofile
flag from docker-compose command.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both plugins called urllib.request.urlopen directly, bypassing the
proxy. Switch to derp.http.urlopen and update the SearXNG endpoint
to the public domain (searx.mymx.me). Update test mocks to match.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bridge networking can't reach the host's loopback. Switch to
network_mode: host so the container shares the host network stack
and can reach the SOCKS5 proxy at 127.0.0.1:1080. Revert proxy
address back to 127.0.0.1.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Container can't reach 127.0.0.1 on the host. Use the host's LAN
address 192.168.129.11 so containerized plugins can reach the
SOCKS5 forwarder.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All plugins importing derp.http failed to load because PySocks
was missing from the container. Add it alongside maxminddb.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Was incorrectly set to 127.0.0.1. The Tor DNSPort runs on the
remote relay at 10.200.1.13:9053. Alt relays noted in comments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both plugins duplicated wire-format helpers and queried the system
resolver on port 53. Switch to shared derp.dns helpers and point
queries at the local Tor DNS resolver (127.0.0.1:9053) so lookups
go through Tor like all other outbound traffic.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bind-mount src/ and data/ alongside plugins/ and config so the
container picks up code changes without rebuilding. Update Makefile
targets, compose file, and INSTALL.md to match.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract shared DNS wire-format helpers into src/derp/dns.py so both
the UDP plugin (dns.py) and the new TCP plugin (tdns.py) share the
same encode/decode/build/parse logic.
The !tdns command routes queries through the SOCKS5 proxy via
derp.http.open_connection, using TCP framing (2-byte length prefix).
Default server: 1.1.1.1.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add create_connection and open_connection helpers to the shared proxy
module, covering portcheck, whois, tlscheck, and crtsh live-cert check.
UDP-based plugins (dns, blacklist, subdomain) stay direct.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add PySocks dependency and shared src/derp/http.py module providing
proxy-aware urlopen() and build_opener() that route through
socks5h://127.0.0.1:1080. Subclassed SocksiPyHandler passes SSL
context through to HTTPS connections.
Swapped 14 external-facing plugins to use the proxied helpers.
Local-only traffic (SearXNG, raw DNS/TLS sockets) stays direct.
Updated test mocks in test_twitch and test_alert accordingly.
Add standalone !searx command for on-demand SearXNG search (top 3 results).
Add SearX as a third backend (sx) to the alert plugin for keyword monitoring.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Search keywords across YouTube (InnerTube) and Twitch (GQL)
simultaneously, announcing new results per channel. Supports
add/del/list/check subcommands with per-platform seen lists.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Poll Twitch streamers via public GQL endpoint and announce
offline-to-live transitions in IRC channels. Tracks stream ID
to avoid re-announcing the same stream.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Follow YouTube channels via Atom feeds with !yt follow/unfollow/list/check.
Resolves any YouTube URL to a channel ID, polls for new videos, and
announces them in IRC channels.
Subscribe RSS/Atom feeds to IRC channels with periodic polling,
new-item announcements, deduplication, and persistence across restarts.
Supports conditional HTTP requests (ETag/Last-Modified), automatic
backoff on errors, and per-channel feed limits.
WHO doesn't support multiple targets (absent from TARGMAX on all
major IRCds). Replace per-nick WHO with a debounced per-channel WHO:
on JOIN, schedule WHO #channel after 2s delay. Subsequent JOINs
within the window reset the timer, so a netsplit producing dozens
of JOINs results in a single WHO.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously the bot only sent WHO on connect (001), so users joining
after the initial scan were never checked for oper status. Now sends
WHO <nick> on every JOIN event to detect opers mid-session.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Calendar reminders use bot.state (SQLite KV) for persistence across
restarts. Supports one-shot at specific date/time and yearly recurring
reminders with leap day handling. Restored automatically on connect
via 001 event handler.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Expand test coverage from 23 pure helper tests to 53 tests covering
the full plugin: _cleanup, _remind_once, _remind_repeat, and the
complete cmd_remind handler (usage, oneshot, repeating, list, cancel,
target routing). Fix IndexError on `!remind every` with no arguments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Supports duration parsing (5m, 1h30m, 2d12h), short hex IDs for
tracking, list/cancel subcommands, and repeating intervals via
`!remind every <duration> <text>`. Includes 23 unit tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Same importlib fix as test_username.py -- load plugins.crtsh from
file path since plugins/ is not a Python package.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Load plugins.username via importlib.util.spec_from_file_location
since plugins/ is not a Python package on sys.path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Queue-based _MockConnection replaces IRCConnection to test the full
bot pipeline (registration -> dispatch -> handler -> response) without
network I/O. 14 tests cover CAP negotiation, PING/PONG, command
dispatch, prefix matching, admin enforcement, channel filtering,
and CTCP responses.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add !username section to USAGE.md with examples. Add OSINT quick
reference entries to CHEATSHEET.md. Mark username plugin done in
ROADMAP.md and TASKS.md.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
53 tests covering regex validation, service registry integrity,
classification logic (status/json/body), formatting, and category
grouping.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cross-platform username OSINT across ~25 services (GitHub, GitLab,
Reddit, Docker Hub, Keybase, Dev.to, Twitch, Steam, etc). Hybrid
approach using HTTP status probes, JSON APIs, and body search.
8 parallel workers via ThreadPoolExecutor, 20s overall timeout.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
!help now only lists commands from plugins allowed in the current
channel. !help <cmd> and !help <plugin> return "unknown" for
filtered plugins. PMs remain unrestricted.
