User-agent: * Disallow: /cgi-bin/ Disallow: /admin/ Disallow: /page_files/ #Sitemap: http://www.yoursite.com/sitemap.gz