relenvant Links
Here some relevant sites which were the basic of Spider Trap.
- Bots List (german)
- Site indexed or only spidered ? (german)
- Robots Experiment (german)
- Robots Standard (english)
- Googlebot informations (english)
- bot-trap without htaccess (english)