Crawl-delay: 10

I think that this directive can be removed from robots.txt, as it is no longer supported.

Google: https://support.google.com/webmasters/thread/13075347

Google announces that they don’t follow this rule from 1st September 2019

Yandex: https://yandex.ru/support/webmaster/robot-workings/crawl-delay.html

On February 22, 2018, Yandex stopped taking into account the Crawl-delay directive.

Please prepare a pull request for. It is the fasted way for us

Best regards Jaap

1 Like

I dont know how to do that… :roll_eyes:
Did not work with github.

Neither did I. But my interest in Hestia motivated me to learn how to use GitHub :smiley: I’m still not “fluent” in Pull Requests though, but practice makes perfect :star_struck: I just wish my day job was not that demanding so I could issue more PRs. Anyway…

It seems like a daunting task at first, but just have patience and you’ll get there.

2 Likes