Duplicate Content

Duplicate
Content Architecture

Multiple URLs serving the same content is splitting your ranking authority.

Where to find it: Screaming Frog > Content > Near Duplicates | Search Console > Indexing > Duplicate, submitted URL not selected as canonical

What It Is

Duplicate content arises from technical causes: HTTP versus HTTPS variants, www versus non-www, trailing slash variations, UTM parameters, print-friendly versions, mobile subdomains, and CMS-generated tag, category page, and excerpt archive duplicates. Each duplicate URL dilutes authority that should concentrate on the canonical version. When the same content exists at multiple URLs, Google must decide which to index and rank — and often makes the wrong choice. Authority from external links pointing to variant URLs never fully consolidates.

Why It Matters

Fixing duplicate content architecture often produces immediate ranking improvements from authority consolidation alone — without any content improvement. Every external link pointing to an HTTP variant, a www variant, or a print page is authority that Google hasn't fully credited to the canonical URL. Resolving these technical duplicates consolidates that fragmented authority onto the canonical pages where it produces ranking benefit. For established sites, the authority consolidation from fixing fundamental URL variants is often the single largest technical SEO win available.

Root Diagnostics

Common Causes

Understanding why this failure occurs is the first step to fixing it permanently.

HTTP and HTTPS Both Resolving

Both HTTP and HTTPS versions of pages accessible without a redirect — splitting link authority between the two protocol variants and creating duplicate indexing issues across the entire site.

www and non-www Both Accessible

Both www.site.com and site.com resolving without redirecting to a single canonical domain — doubling every page's URL surface and confusing Google's entity model for the domain.

CMS-Generated Archive Duplicates

WordPress and similar CMS platforms generating print pages, RSS feed versions, excerpt-only archives, author archives, and date archives — each creating additional indexed versions of content that already exists at canonical URLs.

URL Parameter Tracking Variants

UTM parameters, session IDs, and affiliate tracking parameters creating hundreds of near-duplicate URL variants from a single canonical page — especially problematic when these parameter URLs are linked from external sources.

Interactive Standard Operating Procedure

The Fix Blueprint (Interactive SOP)

Check off each step to monitor your implementation progress live!

Implementation Progress: 0% Completed (0/7)

1. Run Near Duplicates Report in Screaming Frog

Crawl with Screaming Frog and run the Content > Near Duplicates report. This identifies pages with identical or near-identical content at different URLs. Document the most common duplicate patterns to prioritize fixes.

2. Fix HTTP/HTTPS and www/non-www Resolution First

These two fixes affect every single page on the site and are the highest-priority duplicate content fixes. Implement 301 redirects so all HTTP requests redirect to HTTPS and all www/non-www variants redirect to a single canonical domain. Confirm the redirects work for both the homepage and all internal pages.

3. Audit Canonical Tags Sitewide

Export all canonical tags from Screaming Frog. Confirm every page has a self-referencing canonical tag. Identify any pages with missing canonicals, incorrect canonicals pointing to the wrong URL, or variant pages missing canonicals pointing to the preferred version.

4. Identify Parameter-Generated Duplicates in Search Console

Use Search Console's URL Parameters tool to identify which parameters create unique content versus near-duplicates. Configure parameter handling for duplicate-generating parameters — either blocking them in robots.txt or setting canonical tags.

5. Noindex Low-Value CMS Archive Types

For CMS-generated archives that add no unique value — date archives, author archives, tag pages with 1–2 posts — add noindex to remove them from Google's quality assessment. Don't noindex category pages with multiple posts that have topical value.

6. Add Canonical Tags to Print Pages and Feed URLs

For print-friendly page versions and RSS/Atom feed URLs: add canonical tags pointing to the main article page, or disallow in robots.txt. Print pages especially tend to accumulate external links that should consolidate to the main article canonical.

7. Monitor Search Console Duplicate Report

After implementing fixes, track the Indexing > Duplicate URLs report in Search Console. The count should decline over the following 28 days as Google processes the canonical signals and redirect structures. Flag any categories of duplicates that aren't declining.

Tools

Screaming Frog
Paid/Free tier | Near duplicate detection and canonical tag audit — the primary tool for identifying all duplicate content patterns
Siteliner
Free | Alternative duplicate content detection with percentage similarity scoring — useful for identifying high-similarity non-exact duplicates
Google Search Console
Free | Canonical and coverage reports showing which URLs Google has selected as canonicals and which are flagged as duplicates

Time to Fix

2–3 hours

Audit

Variable by scope

Implementation

Pro Tip

Fix HTTP/HTTPS and www/non-www first — they affect every page on the site.

These two fixes are simple 301 redirects but have outsized impact because they affect the entire domain at once. Every external link pointing to an HTTP URL, and every external link pointing to the wrong www variant, is contributing authority that isn't fully credited to the canonical domain. Fixing these two issues consolidates all that fragmented authority onto the correct version of every page simultaneously — making them the highest-ROI duplicate content fixes on almost every site.

Ep 6: Pagination Problems Ep 8: Thin and Low-Value Pages

DuplicateContent Architecture