Impolite Spiders

For years now I have been aghast at the terribly impolite behavior of some bots who spider my site. It is extremely common to see many successive requests to the...