top of page
  • Writer's pictureBrent Payne

Isolated URL only found via a noindex,follow

The discussed URL is isolated, accessible only by following a link with a noindex,follow robots directive.


Why is this important?

Such a scenario presents a URL that is discoverable by search engine crawlers solely through a link with a noindex,follow directive:


<meta name="robots" content="noindex,follow">


This peculiar situation signals to search engines not to index the page, yet still requests them to follow its links.


John Mueller from Google clarified during a 2017 webmaster session how they interpret a noindex, follow directive:

"With noindex, follow, at first, we don't display the page in search results but keep it in our index. However, if the noindex remains over time, we deem that the page should not be used in search and subsequently remove it entirely, ceasing to follow the links – meaning that noindex, follow eventually equates to noindex, nofollow over time."


As a result, if Google stops crawling links to Page B, and if Page C does not garner any other incoming links, it becomes excluded from the web graph. This leads Google ultimately to consider the URL as if it had a noindex,nofollow tag, making it excluded from the structure of the website.


Consequently, this page's capability to rank in search results will diminish and may ultimately be excluded from the indexing process entirely.


What does the Optimization check?

This Optimization is activated when an internal URL obtains no incoming internal links with one exception: links from pages labeled with noindex,follow.


Examples that trigger this Optimization:

This URL receives all its incoming internal links exclusively from pages tagged with:


noindex,follow in the <head>:


<meta name="robots" content="noindex,follow">


OR inside the HTTP header:


X-Robots-Tag: noindex,follow

This Optimization would be active for this URL: https://example.com/page-c


How do you resolve this issue?

If pages become isolated in this manner, their performance in search rankings will decline over time. The essential takeaway from Mueller's statement is:


"...A prolonged noindex will lead us to consider the page should not be indexed, resulting in its full removal. Without the follow, these pages become equivalent to having a noindex, nofollow tag in the long run." 


Once Google ceases to follow links to this page, its ability to rank will suffer.

Rectifying the issue involves removing the noindex directive from relevant pages, or alternatively, establishing additional links within your site's structure to the page, thereby maintaining its status within Google's link graph.

5 views

Recent Posts

See All

ClubReq Gets A Link (Because They Asked)

I am a mentor for Techstars and have been for over 10 years. In those ten years I have mentioned to startups to ask ANYONE and everyone that they meet to link to their site. Yet, in all those times on

Comments


bottom of page