History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“It would be naive to think that peace and justice can be achieved easily. No set of rules or study of history will automatically resolve the problems.... However, with faith and perseverance,... complex problems in the past have been resolved in our search for justice and peace. They can be resolved in the future, provided, of course, that we can think of five new ways to measure the height of a tall building by using a barometer.”
—Jimmy Carter (James Earl Carter, Jr.)
“The history of men’s opposition to women’s emancipation is more interesting perhaps than the story of that emancipation itself.”
—Virginia Woolf (1882–1941)
“Don’t you realize that this is a new empire? Why, folks, there’s never been anything like this since creation. Creation, huh, that took six days, this was done in one. History made in an hour. Why it’s a miracle out of the Old Testament!”
—Howard Estabrook (1884–1978)