Architecture
The List has three parts. The backend (Python, in backend/app) ingests Steve’s List into a SQLite
database and exports it to JSON. The frontend (Astro, in frontend/src) builds a static site from that export.
The offline jobs (GitHub Actions) check Steve’s page every hour, run the backend when it has changed and publish
the site to GitHub Pages. Nothing runs on a server: the published site is static files, plus Supabase for sign-in and Saved
Filters (see ADR 0002
and ADR 0003).
Frontend
At build time
astro build reads the export (shows.json, venues.json, bands.json, meta.json) once, through
lib/siteData.server.ts, and writes every page ahead of time. Each Show, Venue and Band page is built at its Slug,
and again at its numeric URL, which redirects to the Slug. The Alerts, Unsubscribe and 404 pages and
robots.txt need no data, and the docs pages come from the guides’ Markdown, through
Starlight (ADR 0005). The export itself isn’t published.
flowchart TD
accTitle: The site at build time
accDescr: lib/siteData.server.ts reads the export JSON for the Shows list, the Show, Venue and Band pages, the .ics files and home-shows.json; Starlight turns the guides' Markdown into the docs pages. astro build writes every page to frontend/dist, with the Band photos, and the Deploy workflow uploads it to GitHub Pages.
export[("frontend/export<br/>shows, venues, bands, meta")]
guides[("docs/guide/*.md")]
siteData["lib/siteData.server.ts"]
docsLib["Starlight<br/>content.config.ts"]
dataPages["Shows list, Show, Venue<br/>and Band pages, .ics files,<br/>home-shows.json"]
docsPages["Docs pages"]
photos[("Band photos<br/>data branch")]
dist[("frontend/dist")]
host["GitHub Pages"]
export -->|"JSON"| siteData --> dataPages
guides -->|"Markdown"| docsLib --> docsPages
dataPages & docsPages -->|"astro build"| dist
photos -->|"public/images"| dist
dist -->|"upload-pages-artifact"| hostIn the browser
Pages arrive as finished HTML. React runs only in the islands, the parts that change in the browser
(ADR 0004). Two small
scripts without React cover the rest: the inline <head> script hides Shows that have passed since the build, and
lib/keepFilters carries the Shows list’s filters onto Venue links and the Back link. The browser talks to
Supabase directly; its row-level security keeps each person’s Saved Filters private.
flowchart LR
accTitle: The site in the browser
accDescr: A page from GitHub Pages runs the inline past-Shows script, the keepFilters script and its React islands; docs pages with a diagram also load Mermaid. HomeShows fetches home-shows.json. HomeShows, AlertBell and AlertsPage call Supabase for sign-in and Saved Filters; UnsubscribePage calls its unsubscribe function.
page["Page from<br/>GitHub Pages"]
head["<head> script<br/>hides past Shows"]
keep["lib/keepFilters<br/>filters onto links"]
diagrams["lib/diagrams, search<br/>docs pages only"]
subgraph islands ["React islands"]
homeShows["HomeShows<br/>client:load"]
bell["AlertBell<br/>client:idle"]
alertsPage["AlertsPage<br/>client:only"]
unsubscribe["UnsubscribePage<br/>client:only"]
end
homeJson[("home-shows.json")]
supabase[("Supabase<br/>auth, saved_filters")]
mermaid[("mermaid")]
page --> head & keep & diagrams & islands
diagrams -->|"import()"| mermaid
homeShows -->|"fetch"| homeJson
homeShows & bell & alertsPage --> supabase
unsubscribe -->|"rpc"| supabaseBackend
Modules
Everything runs from the command line (python -m app.cli). The GraphQL API (FastAPI and Strawberry) is for
exploring the data locally; the published site never calls it. The export and the Alerts reuse its code (the
mappers and query_shows in graphql/queries.py), so they can’t drift from what the API returns.
flowchart TD
accTitle: The backend's modules
accDescr: cli.py runs the ingest (scheduler.py) and the backfill, the export and the alerts. The ingest and the backfill call the services through ingestion/ and pipeline/, and embed with Ollama into FAISS. The export and the alerts reuse the GraphQL code, as does the local FastAPI app. Everything reads and writes SQLite through the SQLAlchemy models.
cli["app/cli.py"]
api["app/main.py<br/>FastAPI, local only"]
jobs["scheduler.py,<br/>backfill.py"]
exportPy["export.py"]
alertsPy["alerts.py"]
services["ingestion/,<br/>pipeline/"]
embed["pipeline/<br/>embed.py"]
gql["graphql/<br/>Strawberry"]
catalog["catalog.py"]
models["models/"]
ollama["Ollama"]
faiss[("FAISS")]
db[("SQLite")]
out[("export/<br/>*.json")]
cli --> jobs & exportPy & alertsPy
api --> gql
jobs --> services & embed
exportPy & alertsPy --> gql
exportPy --> out
gql --> catalog
services & embed --> ollama
embed & catalog --> faiss
jobs & gql --> models
models -->|"SQL via<br/>SQLAlchemy"| dbcli.pyhas eight commands:check,ingest,export,alerts,notify-success,notify-failure,backfillandphoto-focus.ingestion/andpipeline/enrichment.pycall the services (Steve’s page, Google Places, MusicBrainz, Last.fm, Discogs, Wikimedia Commons, Bandcamp). Ollama runs two models:nomic-embed-textfor embeddings, andtev1:0.8b, which judges whether a new tag is a genre (ingestion/genre_filter.py).catalog.pyholds the read-only queries the API and the export share: filter options, Similar Bands (nearest in the Band FAISS index) and the latest edition’s title.ingestion/band_photos.pysaves Band photos toimages/(≤800px WebP, with a focal point from the faces in each,photo_focus.py).alerts.pyreads Supabase and sends through Resend (see Offline jobs).notifications.pyemails the maintainer how the ingest went, also through Resend (mailer.py).
Data
The core entities (a few fields each), as the SQLAlchemy models in backend/app/models/ store them. A Show is at one Venue and has
its Acts in order (position 0 is the headliner); each Act is one Band’s appearance. A Band’s and a Show’s
embedding is also in its FAISS index, under its id. Two tables stand alone: ingestion_runs (one row per ingest,
with its counts) and genre_tags (whether a tag is a genre, as judged once).
erDiagram
accTitle: The core entities
accDescr: A Venue hosts many Shows. A Show has many Acts, in order. A Band appears as many Acts.
VENUE ||--o{ SHOW : hosts
SHOW ||--o{ ACT : "has, in order"
BAND ||--o{ ACT : "appears as"
VENUE {
int id PK
string name
string city
enum region
}
SHOW {
int id PK
date date
int venue_id FK
time door_time
enum status
bool is_recommended
}
ACT {
int id PK
int show_id FK
int band_id FK
int position "0: headliner"
}
BAND {
int id PK
string name UK
list genres
string image_url
datetime enriched_at
}Saved Filters and Alert subscriptions aren’t in this database: they live in Supabase (saved_filters,
alert_subscriptions; see Weekly Alerts).
Offline jobs
When the Deploy workflow runs
The Deploy workflow (.github/workflows/deploy.yml) has five jobs: check, ingest, build, deploy and
alerts. An ingest happens when the hourly check finds the List changed, and on a manual run that doesn’t skip it;
Alerts go out only after the ingest of a new edition. Runs share the pages concurrency group with the Backfill
workflow, so only one runs at a time.
- The hourly cron:
17 * * * *. Thecheckjob compares Steve’s page with the last one ingested; an unchanged page ends the run there. - By hand: Run workflow, or
gh workflow run deploy.yml, with or without Skip ingestion. - A push to
mainthat touches the site, the backend’s app or the guides. - The Backfill workflow, after it commits to the
databranch.
See Deployment for each.
flowchart TD accTitle: When the Deploy workflow runs accDescr: The hourly cron and a manual run start the check job; a changed List, or any manual run, goes on to the ingest job, and an unchanged List on the cron ends the run. A manual run that skips ingestion, a push to main and the Backfill workflow start at the build job. Build is followed by deploy, and deploy by alerts, but only after the ingest of a new edition. cron["Hourly cron"] manual["By hand"] check["check<br/>cli check"] done(["unchanged:<br/>nothing to do"]) ingest["ingest<br/>cli ingest"] push["Push to main"] backfillWf["Backfill<br/>workflow"] build["build<br/>cli export, astro build"] deploy["deploy<br/>GitHub Pages"] alerts["alerts<br/>cli alerts"] cron --> check manual -->|"ingest"| check check -->|"changed, or by hand"| ingest check -->|"unchanged"| done ingest -->|"commit to data"| build manual -->|"skip ingest"| build push --> build backfillWf --> build build --> deploy deploy -->|"after a new edition only"| alerts
A new edition
Steve updates his page ahead of each Friday’s edition, and corrects it in between. The next hourly check finds the page changed and ingests it. A page whose header date is in a later week (Friday to Thursday) is a new edition, and its run ends by sending Alerts; a correction to the current edition is ingested and deployed the same way, without Alerts.
sequenceDiagram accTitle: A new edition accDescr: The check job fetches Steve's page and finds it changed. The ingest job reads the page, looks new Venues and Bands up on the services and commits to the data branch. The build and deploy jobs publish the site to Pages. The alerts job reads Saved Filters from Supabase and sends each person's Alert through Resend. participant Steve as stevelist.com participant GH as Deploy participant Svc as Services participant Data as data participant Pages participant SB as Supabase Note over GH: check job, hourly Steve->>GH: the List (changed) Note over GH: ingest job Steve->>GH: the List GH->>Svc: new Venues,<br/>new Bands Svc->>GH: genres, links,<br/>photos GH->>Data: commit Note over GH: build, deploy Data->>GH: the data GH->>Pages: the site Note over GH: alerts job,<br/>new edition only SB->>GH: Saved Filters Note over GH: sends each Alert<br/>through Resend
The services are Google Places (new Venues), and MusicBrainz, Last.fm, Discogs, Wikimedia Commons and Bandcamp (new Bands). How ingestion works has the ingest step by step, with what comes from each. The Backfill workflow looks Bands ingested before a lookup existed up on the same services (see Deployment).