Menu

Three event sites already hand you their data. Here's where each one hides it πŸ—‚οΈ
πŸ“°

Three event sites already hand you their data. Here's where each one hides it πŸ—‚οΈ

DEV Community: nodeΒ·Timothy KelvinΒ·9 days ago
#ITPMJ0ya
#devtonodejs#devto#dev#event#events
Reading 0:00
15s threshold

No CSS selectors, no headless browser. The data was in the HTML all along.

This week I built one that collects upcoming events in a city from Eventbrite, Meetup and Luma and puts them in one list. I expected three scrapers full of fragile selectors. I ended up writing almost none, because all three sites already ship their event data inside the page, in a form a program can read. Each one just keeps it somewhere different.

🚦 First, the rules

Before writing any code I read each site's robots.txt, keeping only the rules that apply to every bot ( User-agent: * ):

Eventbrite blocks its RSS and Atom feeds, checkout links and a few API paths. The city search pages ( /d/... ) and event pages are open.

Meetup blocks its API paths ( /api , /meetup_api ) and its feeds. The search page ( /find/ ) and event pages are open.

Luma 's robots.txt only has rules for Googlebot.

robots.txt isn't the whole story, so read each site's terms as well before you build on it.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.