History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“Humankind has understood history as a series of battles because, to this day, it regards conflict as the central facet of life.”
—Anton Pavlovich Chekhov (1860–1904)
“The one duty we owe to history is to rewrite it.”
—Oscar Wilde (1854–1900)
“Revolutions are the periods of history when individuals count most.”
—Norman Mailer (b. 1923)
Related Phrases
Related Words