TL;DR: Log in once in code and reuse the session. For a plain form login, that's a requests.Session that posts the form with its CSRF token. For a JavaScript login or 2FA, use Playwright, log in (by hand if needed) and save storage_state to a file. Then, before you write a single CSS selector, open DevTools and look for the JSON API the logged-in page calls. Requesting that with the same session is usually less code and breaks less often.

First, the boring but important bit

I only do this on my own apps, for my own account's data, or on sites whose terms allow automated access. Read robots.txt and the terms, go slowly, and check for an official API or export first. Don't share credentials, and don't try to get around CAPTCHAs or paywalls.…