A scraper almost never fails the way a program is supposed to fail. It does not raise. It returns a number.
Last week a price list came back as 0 . The page had been redesigned: the price was still on it, in a <span> that no longer carried the class the extractor looked for. The extractor found nothing, applied .get_text() to nothing, and the pipeline wrote 0 into a table, where 0 is a legal price. Nobody saw the problem until a quote went out at zero.
That is the failure mode worth engineering against, and it is not the one most scraping guides prepare you for. An exception is a gift: it stops, it names a line, it leaves a hole you notice.…