History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“Racism is an ism to which everyone in the world today is exposed; for or against, we must take sides. And the history of the future will differ according to the decision which we make.”
—Ruth Benedict (1887–1948)
“The disadvantage of men not knowing the past is that they do not know the present. History is a hill or high point of vantage, from which alone men see the town in which they live or the age in which they are living.”
—Gilbert Keith Chesterton (1874–1936)
“There is nothing truer than myth: history, in its attempt to “realize” myth, distorts it, stops halfway; when history claims to have “succeeded” this is nothing but humbug and mystification. Everything we dream is “realizable.” Reality does not have to be: it is simply what it is.”
—Eugène Ionesco (b. 1912)