Spurious Let's Encrypt SSL Failures 404

Community Support Web
1

We just moved to a fresh HestiaCP VM installed on Debian 11.
Lots of things work better, partly due to a faster VPS provider, but we have one issue: We keep receiving issues with SSL.

When we try to enable SSL on a domain, including the system domain (v-add-letsencrypt-host), we usually get this error:

Error: Let's Encrypt validation status 400 (DOMAIN). Details: 403:"IP: Invalid response from http://DOMAIN/.well-known/acme-challenge/YlVutOVBmmUvUqcyJuQ2zLEF4OElZchYl7s1krczQO4: 404"

A classic, I know. Here is where it gets odd: After trying multiple times over a few days, at some point it does work, including for the host certificate.
But even more odd: Once the certificate is issued, Hestia usually takes a day or two until providing it on the site, redirecting to HTTP until then.

I can not remember such spurious failures from the old system - usually it worked, and when it did not there was an issue to solve.
But this time I had no success analyzing logs so far.

2

Message also appears in the domain log under web/DOMAIN/logs/DOMAIN.log - which seems odd to me, I thought LE requests are served by temporarily resolving via a separate nginx conf?

3

What do you see if you visit:

http://DOMAIN/.well-known/acme-challenge/YlVutOVBmmUvUqcyJuQ2zLEF4OElZchYl7s1krczQO

It should return an code and nothing else …

4

All requests are still logged to access log file …

5

for another domain it returns a code, but for example for one of my domains it returns page not found:
http://melonion.de/.well-known/acme-challenge/_ZTK-fhVQEbYmTVTld964tHdNRLCFmj6vE-TvYTNMEI

Meanwhile it works for https://melonion.me/.well-known/acme-challenge/YlVutOVBmmUvUqcyJuQ2zLEF4OElZchYl7s1krczQO

both pages have the unmodified default webspace

6

It works now for both of them …

7

yeah as said, it took a few days for each domain

might have been related to DNS Cluster Sync incomplete - #5 by eris