top of page
  • Writer's pictureBrent Payne

Noindex in HTML and HTTP header

The URL in question is labeled with a noindex command in both the HTML of the page and within the HTTP header that accompanies it.


Why is this important?

Adhering to a standard where robots directives are only included once per URL minimizes the possibility of errors. It ensures clarity and reduces the chance for human mistakes within the configuration.


In this case, a noindex directive is present both as an X-Robots-Tag in the HTTP header and as a meta noindex within the HTML <head> section of the page.

Although these instructions currently match, it could lead to complications if there's an attempt to alter the page's index status in the future. Should you alter the HTML meta noindex but neglect the HTTP header directive, your page would inadvertently continue instructing search engines not to index it.


As per Google's official guidance, conflicting robots directives will result in search engines opting for the more restrictive option, a pattern which is likely adopted by other search engines as well.


To preclude such issues, it's essential to centralize the location of your robots directives.


What does the Optimization check?

This Optimization will be activated by any internal URL which features noindex directives in both the HTML and HTTP header components.


Examples that trigger this Optimization

This Optimization is set off by any URL that manifests both of the conditions below;


Meta noindex in the <head>:

<!doctype html><html lang="en"><head>  <title>Example Title</title>  <meta name="robots" content="noindex,nofollow">  ...</head><body>...</body></html>


AND also in the HTTP header:

HTTP/... 200 OK...X-Robots-Tag: noindex


Why is this Optimization marked 'Potential Issue'?

This Optimization signifies a 'Potential Issue', which is to say it's currently unlikely to impact the site yet warrants scrutiny as it may lead to complications down the line.


Setting robots directives more than once tends not to be a conscious choice, and so Loud Interactive flags this to help you mitigate potential future misfortunes. Correcting such issues may necessitate technical assistance for the purpose of modifying page templates, plugins, or HTTP headers – thereby eliminating redundancy and defining robots directives singularly.

0 views0 comments

Recent Posts

See All

CSS file size too large discovered in SEO audit

In our SEO audits at Loud Interactive, we often come across a common yet significant issue that can heavily impact your website's loading speed and overall performance: CSS file size too large. Let's

CSS file size too large discovered in SEO audit

When conducting an SEO audit on your site, one issue that might come up is that your CSS file size is too large. But why does this matter, and what can you do about it? Let's dive in. What's the Issue

H1 tag missing or empty" Discovered in an SEO Audit

At Loud Interactive, we've encountered a common issue during our SEO audits that can significantly impact a website's performance on search engines: the "H1 tag missing or empty" warning. This problem

bottom of page