History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“It gives me the greatest pleasure to say, as I do from the bottom of my heart, that never in the history of the country, in any crisis and under any conditions, have our Jewish fellow citizens failed to live up to the highest standards of citizenship and patriotism.”
—William Howard Taft (1857–1930)
“The principle office of history I take to be this: to prevent virtuous actions from being forgotten, and that evil words and deeds should fear an infamous reputation with posterity.”
—Tacitus (c. 55–117)
“The view of Jerusalem is the history of the world; it is more, it is the history of earth and of heaven.”
—Benjamin Disraeli (1804–1881)