Short answer: about 1 in 7 small-business websites aren't fully open to AI crawlers, and most owners didn't choose that. Only about 3% block AI crawlers on purpose in robots.txt.
This is a cross-post of Preferly's October 2026 study . The method, charts and category totals live there, and the aggregate data is free to reuse under CC BY 4.0 .
We checked 4,111 websites of small and local businesses in 26 US cities, and could analyze 3,693 of them. 85.6% were fully open to AI crawlers. The other 14.4% weren't: 5.8% showed our clearly identified crawler a bot check, 5.0% didn't load at all (mostly because of broken HTTPS certificates), 3.1% block some or all AI crawlers in robots.txt, and 0.4% had a robots.txt we couldn't read. Only the robots.txt blocks are a rule written for crawlers. The bot checks come from hosting and security vendors, and the broken homepages are certificate and server errors, so most of those owners never decided anything about AI.…