X-robots-tag in hestia template

[risbo]

Hello friends, need help

I created a duplicated wordpress template and tried to add some x-robots-tag for specific page in nginx server + PHP FPM

location ~* /f/ {
add_header X-Robots-Tag “noindex, nofollow”;
try_files $uri $uri/ /index.php$is_args$args;
}

/f/ should be no index no follow, That are filters from wp. Tried many combinations but no one works.

[sahsanu]

Hello @risbo,

Keep in mind that you should edit both wordpress templates, tpl and stpl.

@risbo

location ~* /f/ {
add_header X-Robots-Tag “noindex, nofollow”;
try_files $uri $uri/ /index.php$is_args$args;
}

Did you copy/paste that conf from some web page? I’m asking because you are using the wrong quotes, you are using “ ” instead of " " so the add_header won’t work.

Use this:

location ~* /f/ {
    add_header X-Robots-Tag "noindex, nofollow";
    try_files $uri $uri/ /index.php$is_args$args;
}

[risbo]

Hi @sahsanu , thanks for answer

Yes i tried both tpl and stpl. Also I used your code and its same issue, x-robots tag is not in header for specific url /f/ .

my tpl: server { listen %ip%:%web_port%; server_name %domain_idn% %alias_idn%; - Pastebin.com
my stpl: #=========================================================================## D - Pastebin.com

[sahsanu]

You should not add two locations with the same pattern (remove one of them):

	location ~ /f/ {
    add_header X-Robots-Tag "noindex, nofollow";
    try_files $uri $uri/ /index.php$is_args$args;
	}

and

	 location ~ /f/ {
        add_header X-Robots-Tag "noindex, nofollow";
    }

Did you rebuild your site after chamging the templates?

[risbo]

I keep first code,(nothing changes) what you mean by rebuild? I change that domain first to default wordpress template and than to custom template, then template should apply right?

location ~* /f/ {
    add_header X-Robots-Tag "noindex, nofollow";
    try_files $uri $uri/ /index.php$is_args$args;
}

[sahsanu]

I keep first code,(nothing changes) what you mean by rebuild? I change that domain first to default wordpress template and than to custom template, then template should apply right?

That should be enought to rebuild your site. Rebuild recreates templates for your domain and put the right conf in the right places.

Check whether your changes have been applied:

/home/YOUR_USER/conf/web/YOUR_DOMAIN/nginx.conf
/home/YOUR_USER/conf/web/YOUR_DOMAIN/nginx.ssl.conf

Also, check headers with another browser or using private/incognito mode. If you tell me what is your domain I could check if I see the header.

Also, in pastebin you are using location ~ /f/ { but now you are using location ~* /f/ {

[risbo]

I checked a nginx.conf and nginx.ssl.conf , location is there. But It not have any effect. I tried both and same issue ~ /f/ and ~* /f/

here is my domain: https://kreatin.rs/c/pre-workout/f/proizvodjaci-activlab/ - Pastebin.com
tpl: Possible Spam Detected - Pastebin.com
smtp: #=========================================================================## D - Pastebin.com

[sahsanu]

I’ve tested it with:

        location ~ /f/ {
                add_header X-Robots-Tag "noindex, nofollow";
                try_files $uri $uri/ /index.php$is_args$args;
        }

And it is working fine:

$ curl -sI https://dnsip.xyz/c/pre-workout/f/proizvodjaci-activlab/ | grep -i robots
x-robots-tag: noindex, nofollow

Did you reload nginx?

systemctl reload nginx

[risbo]

Yes i reload nginx and I can’t see noindex no follow on addon. It says X-robots-tag is not present.

[sahsanu]

I installed a Wordpress to test and this works:

Edit your template and in this block:

		location ~ [^/]\.php(/|$) {
			try_files $uri =404;
 
			include /etc/nginx/fastcgi_params;
 
			fastcgi_index index.php;
			fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
 
			fastcgi_pass %backend_lsnr%;
 
			include %home%/%user%/conf/web/%domain%/nginx.fastcgi_cache.conf*;
 
			if ($request_uri ~* "/wp-admin/|/wp-json/|wp-.*.php|xmlrpc.php|index.php|/store.*|/cart.*|/my-account.*|/checkout.*") {
				set $no_cache 1;
			}
 
			if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in|woocommerce_items_in_cart|woocommerce_cart_hash|PHPSESSID") {
				set $no_cache 1;
			}
		}

add this:

if ($request_uri ~* "/.*/f/.*") {
       add_header X-Robots-Tag "noindex, nofollow";
}

So the final block will look like:

                location ~ [^/]\.php(/|$) {
                        try_files $uri =404;

                        include /etc/nginx/fastcgi_params;

                        fastcgi_index index.php;
                        fastcgi_param HTTP_EARLY_DATA $rfc_early_data if_not_empty;
                        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

                        fastcgi_pass unix:/run/php/php8.1-fpm-dnsip.xyz.sock;

                        include /home/master/conf/web/dnsip.xyz/nginx.fastcgi_cache.conf*;

                        if ($request_uri ~* "/.*/f/.*") {
                                add_header X-Robots-Tag "noindex, nofollow";
                        }

                        if ($request_uri ~* "/wp-admin/|/wp-json/|wp-.*.php|xmlrpc.php|index.php|/store.*|/cart.*|/my-account.*|/checkout.*") {
                                set $no_cache 1;
                        }

                        if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in|woocommerce_items_in_cart|woocommerce_cart_hash|PHPSESSID") {
                                set $no_cache 1;
                        }
                }

[risbo]

@sahsanu

if ($request_uri ~* "/.*/f/.*") {
       add_header X-Robots-Tag "noindex, nofollow";
}

Thank you, this is solution. I appreciate