Google has recently created an additional “Only index content meant for Search” section within the spam guide it updated on Oct. 26. The new section advises websites with pages made for direct sharing, collaboration or utility to set a noindex tag or use a robots.txt directive to block crawlers by default for those pages.

According to Google’s changelog this is an addition relating to “blocking indexing of link-shared content.”

The new section includes a link to the introductory page about robots.txt , and according to that introductory page the robots.txt file “is not a mechanism for keeping a web page out of Google.”

What Google Added

Google’s new “Only index content meant for Search” section follows the guide’s advice on spotting spammy accounts and reads:

“If your service lets users create shareable pages, generated outputs, or interactive tools, consider whether that content is meant to be found through search engines or only accessed directly by people with a link.”

“When pages are created for…