History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“The history of all Magazines shows plainly that those which have attained celebrity were indebted for it to articles similar in natureto Berenice—although, I grant you, far superior in style and execution. I say similar in nature. You ask me in what does this nature consist? In the ludicrous heightened into the grotesque: the fearful coloured into the horrible: the witty exaggerated into the burlesque: the singular wrought out into the strange and mystical.”
—Edgar Allan Poe (1809–1849)
“History takes time.... History makes memory.”
—Gertrude Stein (1874–1946)
“I am ashamed to see what a shallow village tale our so-called History is. How many times must we say Rome, and Paris, and Constantinople! What does Rome know of rat and lizard? What are Olympiads and Consulates to these neighboring systems of being? Nay, what food or experience or succor have they for the Esquimaux seal-hunter, or the Kanaka in his canoe, for the fisherman, the stevedore, the porter?”
—Ralph Waldo Emerson (1803–1882)