The idea
Most metro areas are served by several transit operators, and each publishes its own GTFS bundle and its own GTFS-Realtime feed. Stop IDs collide, route names repeat, timezones and calendar exceptions are handled differently, and one agency's "delayed" is another's missing trip update. This project ingests all of them on a schedule, reconciles stops that are physically the same place, and exposes a single REST endpoint set: nearby stops, departures at a stop, and a trip's live position. Callers never learn how many agencies they are querying.
Why build this
Anyone building a departure board, a kiosk, an e-ink display in a café, a Home Assistant card, or a small mobility app hits the same wall: the raw feeds are publishable but not usable. They spend a week writing a GTFS parser, another week discovering that calendar_dates.txt overrides calendar.txt in ways that break their weekend logic, and then the realtime protobuf turns out to reference trip IDs that were renumbered in last night's static bundle. Transitland and a few agency-specific wrappers exist, but coverage is patchy and realtime is usually the missing piece.
Start with one metro you know well, get it genuinely correct, and sell access. Correctness in this domain is the product — a departure board that is wrong twice a week gets unplugged.
Stack sketch
- Ingest: Python workers using
gtfs-kitfor static bundles andgtfs-realtime-bindingsfor the protobuf feeds. Validate every download with MobilityData'sgtfs-validatorbefore promoting it. - Storage: Postgres with PostGIS. Static data lands in a versioned schema per feed snapshot; a pointer table says which snapshot is live, so a bad bundle is a one-row rollback.
- Stop reconciliation: PostGIS distance plus normalized name similarity (
pg_trgm) to cluster stops across agencies, with a manual override table — the last few percent are judgement calls and should be data, not code. - Realtime: poll each RT feed every 20–30 seconds into Redis with a short TTL, keyed by trip and stop. Never write realtime to Postgres.
- API: FastAPI, cursor pagination, ETags, an API key per caller with per-key rate limits in Redis.
- Ops: Docker Compose behind Traefik, one worker container per feed so a flaky agency cannot stall ingestion.
Scope for v1
- One metro area, every operator in it, static plus realtime.
- Three endpoints:
/stops?lat&lon&radius,/stops/{id}/departures,/trips/{id}. - Daily static refresh with validation and automatic rollback on failure.
- A status page showing, per feed, last successful fetch and validator result.
- Deliberately out: trip planning and routing (that is a different product, and OpenTripPlanner already does it), fares, vehicle occupancy, historical archives, and any write operations.
Where it could go
The obvious expansion is more metros, and the interesting part is that the second one is where you learn which of your assumptions were actually agency-specific. Keep the reconciliation table per metro and the ingest code generic, and each new city becomes a configuration job rather than a rewrite.
After coverage, the two paths are history and prediction. Retain your realtime polls and you own a dataset nobody else has for that city: actual versus scheduled arrival for every trip over months. That supports a reliability API — "how late is this route usually at 08:10 on a Tuesday" — which is more valuable than live position to riders planning a commute, and it is the kind of data local journalists and transit advocates will pay for or cite. Alongside that, a small SDK and a copy-paste departure-board widget turn casual evaluators into paying keys much faster than documentation alone.
Watch out for
Feed licensing varies by agency; some require attribution, a few restrict redistribution, and you are redistributing. Read each license before ingesting and record the terms next to the feed config. The other trap is silent realtime decay — a feed that returns HTTP 200 with stale timestamps looks healthy, so alert on feed age, not on request success.