History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“The only history is a mere question of one’s struggle inside oneself. But that is the joy of it. One need neither discover Americas nor conquer nations, and yet one has as great a work as Columbus or Alexander, to do.”
—D.H. (David Herbert)
“We are told that men protect us; that they are generous, even chivalric in their protection. Gentlemen, if your protectors were women, and they took all your property and your children, and paid you half as much for your work, though as well or better done than your own, would you think much of the chivalry which permitted you to sit in street-cars and picked up your pocket- handkerchief?”
—Mary B. Clay, U.S. suffragist. As quoted in History of Woman Suffrage, vol. 4, ch. 3, by Susan B. Anthony and Ida Husted Harper (1902)
“The steps toward the emancipation of women are first intellectual, then industrial, lastly legal and political. Great strides in the first two of these stages already have been made of millions of women who do not yet perceive that it is surely carrying them towards the last.”
—Ellen Battelle Dietrick, U.S. suffragist. As quoted in History of Woman Suffrage, vol. 4, ch. 13, by Susan B. Anthony and Ida Husted Harper (1902)