History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“History is the present. That’s why every generation writes it anew. But what most people think of as history is its end product, myth.”
—E.L. (Edgar Lawrence)
“It’s a very delicate surgical operation—to cut out the heart without killing the patient. The history of our country, however, is a very tough old patient, and we’ll do the best we can.”
—Dudley Nichols, U.S. screenwriter. Jean Renoir. Sorel (Philip Merivale)
“Considered in its entirety, psychoanalysis won’t do. It’s an end product, moreover, like a dinosaur or a zeppelin; no better theory can ever be erected on its ruins, which will remain for ever one of the saddest and strangest of all landmarks in the history of twentieth-century thought.”
—Peter B. Medawar (1915–1987)