Our car spec site has a bug that hides behind a good excuse. This is the crawl that found it.

Symptom

/toyota/camry/2020/safety renders:

No NCAP data was returned for this model year. The U.S. New Car Assessment Program did not return a record for this vehicle. That is normal for older model years and for vehicles sold in volumes too small to be selected for testing.

Camry, 2020, five stars at NHTSA. The record exists. Our page doesn't have it, and then explains the absence with a reason that isn't true.

Crawl

# 1. all brand sitemaps subs = re . findall ( r " <loc>([^<]+)</loc> " , get ( SITE + " /sitemap.xml " )) # 2. model-year pages only my = [ u for u in pages if re . search ( r " /[^/]+/[^/]+/\d{4}$ " , u )] # 2,792 # 3. classify each safety page " no-record " if " No NCAP data was returned " in text else " rated "

Enter fullscreen mode

Exit fullscreen mode

Sampling: at most three model-years per brand, stratified. 170 pages, 61 brands. /maintenance fetched on all 170, /safety on 162.…