Forward Email

Recent Commits

• titanism (11 Jul 26)

  2.10.3

• titanism (11 Jul 26)

  fix: IMAP correctness, POP3 safety, and server lifecycle improvements IMAP correctness: - Fix on-store websocket notifications to send successfully updated UIDs instead of CONDSTORE conflict UIDs (affected both local and WSP paths) - Fix on-append to set data.draft=true when \Draft flag is added for Drafts - Fix on-move to use correct uid variable after message creation - Fix on-search to handle $gt/$lt operators for date comparisons - Fix on-copy to use push() instead of unshift() for correct UID ordering in UIDPLUS responses (matches on-move behavior and RFC 4315) - Add configurable IMAP notification debounce delays POP3 correctness: - Fix on-update to pass isUid and specific message UIDs to onExpunge, preventing deletion of messages flagged by IMAP but not yet expunged - Add notifier.close() and sqlite_auth_reset unsubscribe on POP3 shutdown SQLite stability: - Rewrite migrate-schema.js for idempotent column/index migrations - Fix $addToSet type validation (was checking property name instead of value) - Add $inc overflow guard in mongoose-to-sqlite updateMany Auth and session: - Fix admin timezone fallback to use correct nested property path - Improve refresh-session error handling and cache invalidation Server lifecycle: - IMAP server close now drains connections and unsubscribes Redis channels - SQLite server close properly awaits database map shutdown Backup reliability: - Fix mbox/eml backup to await output stream close instead of archive end, preventing hash/upload of partially-written files Tests: - Update parse-payload-fixes regression tests

• titanism (10 Jul 26)

  perf(sqlite): VACUUM INTO safety guards, temp DB LRU, debounce reduction VACUUM INTO + atomic rename safety (prevents SQLITE_NOTADB): - Verify new file with setupPragma + quick_check before committing rename - databaseMap.delete() before close to prevent stale handle races - wal_checkpoint(TRUNCATE) before VACUUM to flush WAL data - Recovery path reopens original file on any failure - Clean up stale .vacuum-tmp before starting Performance optimizations: - Temp DB LRU caching (maxSize=100, idleTTL=2m) via temporaryDatabaseMap - Batch eviction (10% of capacity) in DatabaseLRUMap, remove event-loop lag - Main DB cache_size 2MB→64MB (-65536), temp DB override 2MB (-2048) - IMAP notifier debounce 100ms→20ms, max 1000ms→200ms (env-configurable) - WAL checkpoint(PASSIVE) before VACUUM INTO in worker rekey/backup - Unlink tmp backup file after S3 upload to release page cache - Vacuum case in parse-payload → no-op (LRU handles lifecycle) - Reverse DNS → pre-resolved Tangerine hostname with fallback Reliability: - Disable ws.terminate() in sqlite-server ping interval - Suppress ETIMEDOUT/Premature close from error logs and MongoDB - Remove closeDatabase from sync-temporary-mailbox (LRU lifecycle) - Add SQLITE_VERBOSE env flag for Database constructor - Remove DATABASE_MAP_MAX_EVICTIONS_PER_SWEEP env var UI: - Rename "Hard Bounce" → "Rejected" in analytics - Add has_auto_vacuum_migration field to Aliases schema

• titanism (08 Jul 26)

  fix: correct case-insensitive flag comparison in get-forwarding-configuration The condition that auto-adds the 'i' flag for regex aliases without explicit case-insensitive flags compared against '/g:' and '/:', but the REGEX_FLAG_ENDINGS array (line 26) contains values WITHOUT colons ('/g', '/'). The condition never matched, so regex aliases using just '/g' or '/' on the paid plan path were incorrectly treated as case-sensitive. Fix: compare against '/g' and '/' to match the actual array values. This aligns with aliases.js and check-invalid-regex-aliases.js which already use the correct comparison.

• titanism (08 Jul 26)

  2.10.2

• titanism (08 Jul 26)

  fix(imap): remove dead ACK blocking + add SQLITE_DEBUG_TIMERS (console.debug) Remove pRetry/pWaitFor/isRetryableError dead code from imap-server.js onUnpackedMessage handler. The sqlite-server broadcast is fire-and-forget and never consumes ACK UUIDs, making the retry loop dead code that blocked the event listener for up to 15s on transient WS failures. Add SQLITE_DEBUG_TIMERS env var (default true) with console.debug instrumentation across the delivery chain: - sqlite-server: broadcast duration, parsePayload total - parse-payload: getDatabase (stmt), onAppend (append) - imap-notifier: addEntries, fire, scheduleDataEvent - get-database: session.db reuse, cache hit, cache miss - database-lru-map: sweep eviction count + duration - close-database: total close duration

• titanism (08 Jul 26)

  fix: handle all regex flag endings in splitString comma-splitting The splitString function only checked for '/:' when determining where a regex entry ends before splitting on commas. This caused regex aliases with flags (/i, /g, /gi, /ig) AND commas in the pattern (e.g. {0,1}) to be incorrectly split at the internal comma, destroying the regex entry and causing the catch-all to win instead. Now checks all flag endings: /gi:, /ig:, /g:, /i:, /: by finding the earliest (leftmost) occurrence of any ending. When no recognized ending is found, the entire string is treated as a single entry to avoid splitting on commas inside regex character classes.

• titanism (08 Jul 26)

  2.10.1

• titanism (08 Jul 26)

  fix(sqlite): eliminate 100GB+ memory bloat, WebSocket stalls, and IOERR_SHORT_READ Root cause: DATABASE_MAP_MAX_SIZE=3000 with maxEvictionsPerSweep=10 meant up to 3000 SQLite DBs stayed open per worker (3000 x 4MB page cache = 12GB per worker, 48GB across 4 PM2 workers). The sweep could only close 10 DBs/min — 5 hours to drain a full map while new DBs kept opening. The broadcast logic blocked the event loop for up to 5 minutes (pWaitFor ACK loop), preventing sweeps from running. closeDatabase() didn't always close the handle when PRAGMA optimize threw, leaking orphaned DB handles. At cc6c054d (working), the sweep had NO eviction throttle and closed ALL idle DBs every cycle, keeping the actual open count at ~50-200. The regression was introduced in d87098924 which added maxEvictionsPerSweep=10. P0 — Memory bloat (100GB+): - .env.defaults: DATABASE_MAP_MAX_SIZE 3000->500, MAX_EVICTIONS 10->50 - helpers/setup-pragma.js: cache_size 4MB->2MB per DB (500x2MB = 1GB max/worker) - helpers/database-lru-map.js: clean drop-in Map replacement with LRU eviction, idle TTL (5m), inTransaction protection, event-loop lag detection P0 — WebSocket broadcast blocking (5min pWaitFor timeout): - sqlite-server.js: replaced blocking pWaitFor ACK loop with non-blocking fire-and-forget: send to local clients + publish to Redis for cross-worker delivery. Removed randomUUID/uuidsReceived/isRetryableError. Added Redis subscriber in listen() for cross-process broadcast with workerId-based self-echo filtering (prevents local clients from receiving broadcast twice). maxPayload 256MB on WebSocketServer to prevent OOM from malicious frames. P1 — IOERR_SHORT_READ on close: - helpers/close-database.js: PRAGMA optimize disabled entirely (it was the root cause — runs ANALYZE/full table scans during close, causing IOERR_SHORT_READ when another worker is mid-write on the same WAL). Code is commented out with detailed notes explaining why and alternative approaches (run on scheduled worker during low-traffic hours, or on open with 0x10002 flag). Still waits up to 30s for inTransaction to clear before closing. P1 — Eviction of active databases: - DatabaseLRUMap protects active DBs via two mechanisms: 1. get() updates lastAccess — any DB being actively used stays fresh 2. inTransaction check — any DB mid-write is never evicted No ref/unref needed — clean drop-in Map API (get/set/has/delete/keys/size). P2 — tmpDb fallback with databaseMap re-check: - helpers/parse-payload.js: before falling back to tmpDb, re-checks databaseMap in case an IMAP session opened the DB between the initial check and fallback. Avoids unnecessary tmp writes and reduces I/O contention. Fallback success path fires sendApn + sendWebSocketNotification + iMIP processing (matching the main onAppend success path). Tests (44 passing, XO lint clean): - test/helpers/close-database.js (8 tests) - test/helpers/database-lru-map.js (24 tests) - test/helpers/parse-payload-fixes.js (8 tests) - test/helpers/sqlite-server-broadcast.js (4 tests)

• titanism (07 Jul 26)

  fix: fixed touch payload action

• titanism (07 Jul 26)

  2.10.0

• titanism (07 Jul 26)

  fix(sqlite): resolve database I/O storms and LRU eviction issues - Remove dropPageCache from close-database.js (root cause of I/O storms that triggered cascading fadvise calls on every database close) - Increase DATABASE_MAP_MAX_SIZE from 1000 to 3000 to reduce eviction pressure under normal load - Add throttled LRU sweep with DATABASE_MAP_MAX_EVICTIONS_PER_SWEEP=10 to prevent mass-eviction thundering herds - Add reference counting and eviction protection to database-lru-map.js - Add IMAP keepalive interval (4min) that touches connected users' databases to prevent LRU eviction of active sessions - Add 'touch' action to parse-payload.js for LRU lastAccess refresh - Add 'database connection is not open' as a retryable error with lazy-require of get-database in onFailedAttempt to fix circular dep - Disable sqlite_auth_request pub/sub mechanism (auth handled directly) - Remove temporaryDatabaseMap caching (open/close per request instead) - Add backup dedup via Redis SET NX (once per alias per day) - Restore journal_size_limit to 256MB (was incorrectly reduced to 64MB) - Add sent_at timing diagnostic to MX tmp action - Insert welcome email on first-time mailbox setup - Reduce pWaitFor broadcast timeout from 5m to 15s - Update test assertions to account for welcome email (+1 in INBOX)

• titanism (07 Jul 26)

  docs: fixed alignment

• titanism (07 Jul 26)

  docs(readme): expand TLSA section to cover all services with automation - List all 22 TLSA records (MX, SMTP, IMAP, POP3, Web, API, CalDAV, CardDAV) - Add CalDAV and CardDAV TLSA records (previously missing) - Add Cloudflare API one-liner to create all records with a new hash - Add Cloudflare API one-liner to delete old records by hash after grace period

• titanism (07 Jul 26)

  fix(status): filter issues by creator instead of label Replace labels:'status' with creator:'titanism' so any open issue created by titanism triggers the badge/toast/event-feed/ICS regardless of which label is applied.

• titanism (07 Jul 26)

  fix: authenticate all GitHub API calls with GITHUB_OCTOKIT_TOKEN - get-github-releases.js: add Bearer auth to undici.fetch calls - checkGitHubStars: add Bearer auth to undici.request call - check-disposable.js: add Bearer auth to raw.githubusercontent fetch - get-status-incidents.js: rewrite to use @octokit/rest with auth - checkGitHubIssues: add labels/state filters, remove user filter All callers now use authenticated requests (5000 req/hr) instead of unauthenticated (60 req/hr), preventing rate limit failures across 42 worker processes.

• titanism (07 Jul 26)

  fix(logs): add accurate delivery timing to IMAP/webhook logs - Measure messageTime around wsp.request (IMAP) and retryRequest (webhook) - Derive envelopeTime from session.arrivalTime for total delivery context - Populate info.accepted, info.envelope, info.messageSize - Fall back to RCPT TO for destination display on old logs

• titanism (07 Jul 26)

  fix(logs): use contextual label for delivery destination - Show "Forwarded To" when message was forwarded to a different address - Show "Delivered To" when delivered to the user's own IMAP/POP3 mailbox - Rename CSV column header to "Delivered To" (generic, always accurate) - Update API spec descriptions with proper translations in all 25 locales

• titanism (07 Jul 26)

  perf(logs): add MongoDB indexes for message size and delivery time sorting - Add partial index on meta.info.messageSize for sort-by-size queries - Add partial index on meta.info.deliveryTime for sort-by-delivery-time queries - Add pre('save') hook to materialize deliveryTime (envelopeTime + messageTime) - Make Delivery Time column header sortable in logs list table Note: existing documents need a one-time backfill migration: db.logs.updateMany( { 'meta.info.envelopeTime': { $exists: true } }, [{ $set: { 'meta.info.deliveryTime': { $add: [ { $ifNull: ['$meta.info.envelopeTime', 0] }, { $ifNull: ['$meta.info.messageTime', 0] } ] } } }] )

• titanism (07 Jul 26)

  fix: bump max size

• titanism (07 Jul 26)

  Revert "Reapply "fix: improve TTI delivery reliability with six targeted optimizations"" This reverts commit afebf464ea03371562ffed19be8abc28fab352e8.

• titanism (07 Jul 26)

  2.9.9

• titanism (07 Jul 26)

  feat(logs): improve log list/detail UX with delivery destination, timing, and size - Add "Forwarded To" badge showing actual delivery destination in logs list - Add sortable "Size" and "Delivery Time" columns to logs list table - Responsive design: columns hide on mobile with inline badge fallbacks - Color-coded delivery times (green <1s, blue 1-10s, yellow 10-60s, red >60s) - Add Forwarded To, Delivery Time, Message Size rows to log detail page - Show SMTP envelope and message transfer time breakdown on detail page - Add 3 new columns to CSV export: Forwarded To, Delivery Time (ms), Message Size (bytes) - Wrap translatable badge labels in t() for i18n (keep "Forward Email" as brand name) - Replace template literal concatenation with pug interpolation syntax - Update /v1/logs/download API spec description with translated column list in all 25 locales

• titanism (07 Jul 26)

  perf: reduce TTI via sqlite_auth_request, drop page cache on DB close, tune memory

• titanism (07 Jul 26)

  docs: add DANE/TLSA update instructions before certificate deployment

• titanism (07 Jul 26)

  Reapply "fix: improve TTI delivery reliability with six targeted optimizations" This reverts commit 01484e7be04727e696d886e1213e339985fb5f88.

• titanism (07 Jul 26)

  Revert "fix: improve TTI delivery reliability with six targeted optimizations" This reverts commit 999a6f125f08298a509eec6d28130cdcf865c481.

• titanism (07 Jul 26)

  2.9.8

• titanism (07 Jul 26)

  fix: implement :index/:last support in Sieve header tests (RFC 5260)

• titanism (07 Jul 26)

  fix(send-apn): set Mail.pushType to 'background' and expose note.aps + compile() Three related issues in helpers/send-apn.js: 1. SERVICES.Mail.pushType was `undefined` instead of `'background'`. The APNs `apns-push-type` header was being sent as the literal string "undefined" for Mail pushes. All three services (Mail, Calendar, Contact) must use 'background' per the dovecot-xaps-daemon reference. 2. createNote() now exposes `note.aps` as a top-level property on the returned note object: - Mail: populated with `account-id` (when present) and `m` (mailbox hash) - Calendar/Contact: empty `{}` (no aps in wire payload) This separates the metadata view (`note.aps`) from the wire payload (`note.payload`) and enables tests to assert on the aps structure without reaching into `note.payload.aps`. 3. Added `note.compile()` method that returns `JSON.stringify(note.payload)`. The `send()` method now uses `note.compile()` instead of inlining `JSON.stringify(note.payload)`, making the wire serialization testable and consistent. For Mail, `note.payload.aps` still references the same object as `note.aps` so the wire JSON is unchanged. For Calendar/Contact, the wire JSON remains `{ key, dataChangedTimestamp, pushRequestSubmittedTimestamp }` with no aps dictionary, matching Apple's ccs-calendarserver reference implementation.