User-agent: * Request-rate: 1/600 Crawl-delay: 600 #Disallow: /bugs/ # Hide the base folder (no indexing) Disallow: /bugs/browser # Hide browser #Disallow: /bugs/changeset # Hide changeset #Disallow: /bugs/log # Hide log #Disallow: /bugs/milestone # Hide milestone Disallow: /bugs/search # Hide search #Disallow: /bugs/timeline # Hide timeline #Disallow: /bugs/wiki # Hide wiki #Disallow: /bugs/roadmap # Hide roadmap #Disallow: /bugs/report # Hide report #Disallow: /bugs/newticket # Hide newticket