An Attempt to Ban Bad Bots Crawling My Sites

Some features may only be available on the original post . I don't really like bad bots , and by that I mean crawlers that don't care about robots.txt . The reason is simple: I don't want my data fed into obscure systems, and also just by principle, if we give you rules, follow them . Credit where it's due: the idea came from Caolan's website . The idea is simple: make the bad bots click a link…