Menu

πŸ“°
0

An update on the scraper situation

LWN.netΒ·Jonathan CorbetJuly 10, 2026Β·2 months ago
#RIaq8swL
#sites#traffic#article#englishlanguage
Reading 0:00
15s threshold

This article brought to you by LWN subscribers Subscribers to LWN.net made this article β€” and everything that surrounds it β€” possible. If you appreciate our content, please buy a subscription and make the next set of articles possible. Our article " Fighting the AI scraper bot scourge ", published in early 2025, discussed the problem of widespread scraping of web sites in search of training data for large language models and related projects. This activity overwhelms sites with traffic. Over a year after that article is published, the problem is still growing. The hammering of sites by shadowy actors has reached new heights, and the open web is becoming increasingly difficult to maintain. Where is this traffic coming from, and what can be done about it? Residential proxies As was described last year, scraper attacks come from a huge number of sources across the net.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More