# Robots.txt file from http://www.xdnet.net # All robots will spider the domain User-agent: * Disallow: /admin.php/ Disallow: /rich/ Disallow: /images/ Disallow: /web/ Disallow: /cert/ Disallow: /data/sqldata/