2 Respostas
- Mais recentes
- Mais votos
- Mais comentários
0
Hi, find the official doc here on this very item. https://developers.google.com/search/docs/crawling-indexing/reduce-crawl-rate
0
You can use Robots.txt
here is a sample file for you
User-agent: *
Disallow: /private/
Disallow: /temp/
Disallow: /test/
User-agent: Googlebot
Allow: /
Disallow: /example-subdirectory/
Crawl-delay: 10
User-agent: Bingbot
Allow: /
Disallow: /example-subdirectory/
Crawl-delay: 10
# Sitemap link
Sitemap: https://www.yoursite.com/sitemap.xml
respondido há 2 anos
