When you need Letterboxd Film & Review as a recurring feed, the gap between "got a few rows out" and "have a clean nightly dataset in the warehouse" is wider than it looks. Here is the pipeline I sketched out, with the decisions I made at each step.

Source survey

Letterboxd Scraper Films, Ratings, Reviews & User Data Scrape films, ratings, cast & crew, genres, and user reviews from Letterboxd, the world's leading social film-discovery platform. For pipeline purposes, the relevant questions are: how stable is the source markup, what is the natural pagination unit, and how aggressively does it rate-limit.…