This refers to a situation where a URL is functionally equivalent to another indexable URL. These duplicates often stem from variations in character case, or the sequence of query string parameters retaining the same values.
Why is this important?
Such duplications represent a significant problem, creating identical URLs that can be indexed by search engines.
Issues with large scale identical content can trigger search engine quality filters like Google's Panda, which may result in reduced organic search traffic to a website.
What does the Optimization check?
The Optimization will activate for any internal, indexable URL which exactly matches another internal, indexable URL.
Note: The duplicate content check excludes URLs marked as canonical since the canonical link element resolves the problem of duplicate content.
Examples that trigger this Optimization
The Optimization targets any internal, indexable URL that is identical to another, considering various factors.
For instance, two URLs with the same parameters in a different sequence will trigger it:
https://example.com/page/?a=1&b=2https://example.com/page/?b=2&a=1
It will also trigger for URLs differing only by letter casing:
How do you resolve this issue?
The impact of duplicate content hinges on the extent of the problem. A small number of duplicates may be negligible, but thousands can be significant enough to affect site quality scores.
This issue can spiral rapidly if search engines index duplicate URLs widely, making it vital to address.
Solutions vary depending on the issue:
Address unwanted duplication of query strings by working with developers to correct the script responsible for URL generation. Addressing the root cause is preferable to using redirects or canonical tags, which can affect the crawl budget.
If case differences create duplicates, eliminate links to the uppercase URLs to prevent crawling, and employ a 301 redirect as a precaution. If a 301 redirect is not feasible, use a canonical tag as an alternative.
How do you get more data from Sitebulb?
In Sitebulb, one can either inspect duplicates on a per-URL basis or export all duplicate content records for analysis in a spreadsheet.
To inspect a specific URL, click the URL Details button in the URL List view.
To export all duplicate URL data, use the Export Optimization Data button at the top right.
The export will provide a structured Excel spreadsheet, facilitating sorting and review of technically duplicate URLs.