History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“This above all makes history useful and desirable: it unfolds before our eyes a glorious record of exemplary actions.”
—Titus Livius (Livy)
“The history of every country begins in the heart of a man or a woman.”
—Willa Cather (1876–1947)
“It is remarkable how closely the history of the apple tree is connected with that of man.”
—Henry David Thoreau (1817–1862)
Related Phrases
Related Words