# Spider exclusion list instructions
User-Agent: *
Disallow:
.htaccess
/cgi-bin/
/css/
/downloads/
/errors/
/forms/
/GeneratedItems/
/images/
/javascript/
/pages/terms_of_use.html
/pages/privacy_policy.html
/robots.txt