top of page
  • Writer's pictureBrent Payne

Nofollow only in the HTTP response HTML

The issue at hand pertains to URLs that possess a nofollow directive solely within the HTTP response HTML rather than being present in the rendered HTML.


Why is this important?

Webpage rendering demands substantial resources and time compared to simply retrieving the source HTML. Google, therefore, approaches URL crawling in two phases: a preliminary assessment of the response HTML and a subsequent appraisal post-rendering. The search index is then adjusted based on the post-render findings.


At the 'Processing' phase, activities unfold simultaneously, introducing complexity as Google determines which pages to render based on meta robots checks. During this phase, the response HTML is examined for followable links that are fed back into the crawl queue. This happens prior to the rendering phase.


A nofollow directive in the response HTML could decelerate the discovery and indexing of linked URLs by Google.


Moreover, inconsistency in nofollow indications raises ambiguity—should the nofollow also be reflected in the rendered HTML or not?


What does the Optimization check?

Loud Interactive's Optimization is activated by any internal URL that carries a nofollow robots tag in the response HTML but lacks it in the rendered HTML.


Examples that trigger this Optimization

Consider a circumstance where an internal URL has a meta robots=nofollow tag in the <head> of the response HTML but fails to show it in the rendered HTML.


For instance, the response HTML might appear as follows:

<!doctype html><html lang="en"><head>    <meta name="robots" content="index,nofollow">    ...</head><body>...</body></html>


In contrast, the rendered HTML could be:

<!doctype html><html lang="en"><head>    <meta name="robots" content="index,follow">    ...</head><body>...</body></html>


How do you resolve this issue?

Clarification regarding whether pages ought to be marked nofollow is crucial. Align the robots directives in the response HTML with those post-JavaScript rendering to ensure consistency.


Further Reading

0 views0 comments

Recent Posts

See All

CSS file size too large discovered in SEO audit

In our SEO audits at Loud Interactive, we often come across a common yet significant issue that can heavily impact your website's loading speed and overall performance: CSS file size too large. Let's

CSS file size too large discovered in SEO audit

When conducting an SEO audit on your site, one issue that might come up is that your CSS file size is too large. But why does this matter, and what can you do about it? Let's dive in. What's the Issue

H1 tag missing or empty" Discovered in an SEO Audit

At Loud Interactive, we've encountered a common issue during our SEO audits that can significantly impact a website's performance on search engines: the "H1 tag missing or empty" warning. This problem

bottom of page