robots.txt icon. valid for http, https, and ftp protocols. the robots txt file gives recommendations to search

robots txt icon seo crawlable page search engine internet web symbol vector design disallow allow useragent technology concept illustration sign information file protocol document background website flat data crawler scraping index indexing info rep exclusion https http ftp