Not sure if my robots.txt is okay Not sure if my robots.txt is okay
 

News:

cpg1.5.48 Security release - upgrade mandatory!
The Coppermine development team is releasing a security update for Coppermine in order to counter a recently discovered vulnerability. It is important that all users who run version cpg1.5.46 or older update to this latest version as soon as possible.
[more]

Main Menu

Not sure if my robots.txt is okay

Started by macsimoin, July 02, 2006, 04:46:50 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

macsimoin

Since I've registered a new domain and relocated my gallery into the root of the new domain, I added a robots.txt file there as well. Can someone have a look at this and let me know if it looks okay or if it's in the right place?

#
# robots.txt for http://www.kiltphotos.com/ and friends
#

User-agent: Zealbot
Disallow: /

User-agent: MSIECrawler
Disallow: /

User-agent: SiteSnagger
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: Fetch
Disallow: /

User-agent: Offline Explorer
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: WebZIP
Disallow: /

User-agent: linko
Disallow: /

User-agent: HTTrack
Disallow: /

User-agent: Microsoft.URL.Control
Disallow: /

User-agent: Xenu
Disallow: /

User-agent: larbin
Disallow: /

User-agent: libwww
Disallow: /

User-agent: ZyBORG
Disallow: /

User-agent: Download Ninja
Disallow: /

User-agent: wget
Disallow: /

User-agent: grub-client
Disallow: /

User-agent: k2spider
Disallow: /

User-agent: NPBot
Disallow: /

User-agent: WebReaper
Disallow: /

User-agent: Konqueror
Disallow: /

User-agent: msnbot
Disallow:

User-agent: psbot
Disallow:

User-agent: Googlebot
Disallow:

User-agent: Googlebot-Image
Disallow:

User-agent: *
Disallow: /albums/
Disallow: /bridge/
Disallow: /docs/
Disallow: /images/
Disallow: /include/
Disallow: /lang/
Disallow: /logs/
Disallow: /plugins/
Disallow: /README_FILES/
Disallow: /sql/
Disallow: /themes/


Thanks. :)