To disallow all remove # # User-agent: * # Disallow: / # # Allow access to HTML Pages Only # User-agent: * Disallow: /_private Disallow: /_borders Disallow: /cgi-bin Disallow: /feedback Disallow: /pdf Disallow: /images Disallow: /wwwstat