User-agent: Googlebot-Image Disallow: /book/ Disallow: /books/ Disallow: /stock/ Disallow: /cgi-bin/ Disallow: /stock/ # Education bot to help avoid student's plagarism User-agent: TurnitinBot Disallow: / User-agent: QihooBot Disallow: / User-agent: Gaisbot Disallow: / User-agent: gigabot Disallow: / User-agent: Gigabot Disallow: / User-agent: psbot Disallow: / User-agent: VoilaBot Disallow: / User-agent: ichiro Disallow: / # Internet Archive User-agent: ia_archiver Disallow: / User-agent: * Disallow: /cgi-bin/ Disallow: /img/ Disallow: /images/ Disallow: /stock/ Disallow: /admin/ # for a in www.*; do cp www.loanspage.co.uk/web/robots.txt $a/web/robots.txt; done;