Menu

1 / 2

What I learned scraping JSON-LD Schema & Meta Tag Extractor: schema, gotchas and the tooling that worked

DEV Community·Can Yılmaz·6 months ago
#nGr9fbLB
#webscraping#apify#data#spinach#feta
Reading 0:00
15s threshold

I had a short window this week to evaluate JSON-LD Schema & Meta Tag Extractor as a data source. Here is the condensed write-up of what the data looks like, what surprised me, and the bits of infrastructure that paid off.

The source

JSON-LD Schema & Meta Tag Extractor Scrape Schema.org, OpenGraph & Meta Tags Extract structured data and SEO metadata from any webpage in seconds. The relevant questions for any new source are always: is the markup stable, is pagination sensible, and how aggressively does it rate-limit. For this one, all three answers are "good enough that you can build on it" -- which is honestly more than I can say for a lot of supposedly easy targets.

The schema

What you get back per record:

url -- url

pageTitle -- page title

metaDescription -- meta description

jsonLd -- json ld

openGraph -- open graph

twitter -- twitter

scrapeDate -- scrape date

Nothing exotic, which is exactly what you want from a feed.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.