Crawl-delay: 10

I think that this directive can be removed from robots.txt, as it is no longer supported.


Google announces that they don’t follow this rule from 1st September 2019


On February 22, 2018, Yandex stopped taking into account the Crawl-delay directive.

Please prepare a pull request for. It is the fasted way for us

Best regards Jaap

1 Like

I dont know how to do that… :roll_eyes:
Did not work with github.

Neither did I. But my interest in Hestia motivated me to learn how to use GitHub :smiley: I’m still not “fluent” in Pull Requests though, but practice makes perfect :star_struck: I just wish my day job was not that demanding so I could issue more PRs. Anyway…

It seems like a daunting task at first, but just have patience and you’ll get there.