This site does not allow itself to be embedded. You can still read it on the original site — the toolbar below keeps your place in the directory.
I have been blocking AI from scrapping this blog since a while using the robots.txt. You can get a list of user agents at darkvisitors.com. Inspired by Melanie Kat, I added a rewrite rule to feed them bots. <configuration> <system.webServer> <rewrite> <rules> <clear /> <rule name='Give the AI bots some food' enabled='true' stopProcessing='true'> <match url='(.*)' /> <conditions…
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.