An identified URL has been specified as an alternate for hreflang, but it is excluded by the robots.txt file.
Why is this important?
Search engines use hreflang tags as directives for indexing. If a page in English points to a French equivalent via hreflang, the search engine is asked to index both versions for the relevant language audiences.
Should the English version be restricted by the robots.txt file, search engines would be unable to crawl its content. Although they can still index the URL, they won’t be able to detect the alternate URLs or confirm the reciprocal hreflang tags, which are crucial for the correct functioning of hreflang.
What does the Optimization check?
The Optimization will activate for any URL that is an hreflang alternate and has been disallowed.
Note: This Optimization resembles another Optimization: "Has outgoing hreflang annotations to disallowed URLs". The key difference lies in whether the focus is on the target (incoming hreflang) or the source (outgoing hreflang) of the hreflang annotation.
Examples that trigger this Optimization:
Imagine the following URL: https://example.com/us/product/
This Optimization would trigger if the URL was disallowed by robots.txt and if said URL appears as an hreflang alternate on another page or self-references with an hreflang annotation.
<link rel="alternate" href="https://example.com/us/product/" hreflang="en-us" />
<link rel="alternate" href="https://example.com/fr/product/" hreflang="fr-fr" />
How do you resolve this issue?
A discrepancy like this can be challenging to troubleshoot as it's unclear whether the hreflang is incorrect or the robots.txt file needs to be updated.
Determining the intended hreflang URL will require a close examination. After identifying the appropriate URL, the necessary adjustments become clear:
If the hreflang annotation is accurate, then the robots.txt file is at fault and must be edited to change or eliminate the particular rule causing the issue.
If the hreflang annotation is faulty, then these annotations must be updated to reference the correct URL.