The idea
A hosted service that ingests a creator's back catalog — via RSS feed, Substack export, or a Ghost/WordPress API connection — extracts every outbound link from every post, and rechecks them on a recurring schedule. When a link 404s, redirects somewhere unexpected, or drops behind a new paywall, the tool flags it and suggests a fix: an archive.org snapshot, a canonical replacement if the domain moved, or just a clean "removed" marker. Creators get a weekly digest instead of finding out from a reader's reply-all email.
Why build this
Anyone with a few years of published back catalog has this problem and doesn't know it, because nobody re-reads old posts. Link rot compounds silently — studies of long-running blogs put dead-link rates well above 30% after five years — and it quietly erodes trust and SEO (Google penalizes pages with broken outbound links during recrawls). Newsletter writers in particular reference other people's posts constantly, and those posts move, get paywalled, or vanish when a small blog shuts down. There's no dedicated tool for this: general link checkers exist for site QA at build time, but nothing runs continuously against a growing archive and understands newsletter-specific ingestion (Substack export format, RSS with full content, Ghost's admin API).
Stack sketch
- Frontend: Next.js dashboard showing per-post link health, a rot-rate trend line, and a queue of flagged links awaiting action
- Backend: Node/Fastify API; a worker queue (BullMQ + Redis) that rechecks links on staggered schedules (daily for recent posts, monthly for old ones) to avoid hammering any one domain
- Storage: Postgres for posts, links, and check-history; every check result is versioned so trend charts work
- Link checks: HEAD requests first, falling back to a headless fetch (Playwright) for JS-rendered pages or bot-walled sites; a lightweight paywall heuristic (looking for known paywall markup/meta tags)
- Recovery suggestions: Wayback Machine's Availability API for archived snapshots; simple domain-move detection by diffing redirect chains against a known list of blog migration patterns
- Ingestion: RSS/Atom polling, a Substack CSV export importer, and a Ghost Admin API connector for direct sync
Scope for v1
- In: RSS-based ingestion (covers Substack, Ghost, WordPress, most blogging platforms out of the box), scheduled rechecking, a dashboard with a flagged-links queue, Wayback Machine suggestions, weekly email digest
- Out: automatic in-place link fixing (creators should approve replacements manually — silently rewriting old posts is risky), paywall detection beyond a basic heuristic, non-RSS platforms (Medium, LinkedIn articles) until there's demand
- Out: multi-user team accounts — v1 is single-creator, single-archive
Where it could go
The natural extension is citation quality, not just link liveness: flagging when a linked source has been updated or retracted since the original post cited it, which matters a lot for anyone writing about research, news, or fast-moving topics. From there, a browser extension that checks links inline while a creator is drafting a new post — before it's even published — closes the loop. A publisher-facing version could aggregate rot rates across an entire newsletter network (a la a Substack Pro dashboard) as a trust signal for advertisers evaluating placement.
Watch out for
Aggressive recheck scheduling against thousands of external domains looks like scraping abuse; respect robots.txt, rate-limit per domain, and set a clearly identifying user agent, or you'll get IP-blocked by the exact sites you're trying to monitor.