Broken Link Audit Without Paid Tools: How to Find and Fix 404 Errors That Hurt Your SEO

I found 47 broken links on a site I’d been running for six months. Forty-seven. And I had no idea they existed.

The worst part wasn’t the number. It was that three of those broken links were on my most-trafficked blog post, pointing to resources I’d moved during a site restructure. Every visitor who clicked those links hit a 404 page. Every crawler that followed them wasted a request. And every ounce of internal link equity flowing through those anchors evaporated into nothing.

I found them not because I was proactive, but because I ran a free crawl on a whim after reading about crawl budgets. The discovery sent me down a rabbit hole of broken link auditing that completely changed how I maintain my sites. This guide walks through everything I learned, including the Python crawler I built that catches broken links more reliably than any free online tool I’ve tested.

Why Broken Links Matter More Than You Think

Most SEO advice treats broken links as a minor housekeeping issue. “Fix them when you find them” is the common refrain. This understates the problem. Broken links cause three distinct types of damage, and the compounding effect is worse than any single issue suggests.

Damage 1: Wasted Crawl Budget

Googlebot allocates a finite crawl budget to every site, determined by your server’s response speed and the site’s overall size. When Googlebot encounters a 404, it doesn’t just note the error and move on. It logs the URL, schedules a revisit to confirm the page is still gone, and revisits it again weeks later. I’ve seen sites where 30% of crawl requests went to URLs that returned 404, while new content sat uncrawled for days.

On small sites with 50 pages, this barely matters. Googlebot will crawl everything regardless. But on sites with hundreds or thousands of URLs, especially those with dynamic URL parameters or faceted navigation, broken links can literally prevent new pages from being discovered. I watched a site with 300 pages take three weeks to get a new blog post indexed because the crawl budget was consumed by 80 stale URLs from a deleted product catalog.

Damage 2: Leaked Link Equity

Internal links pass link equity. When Page A links to Page B, a portion of Page A’s ranking power flows to Page B. This is one of the few ranking factors you have complete control over.

When Page A links to a URL that returns 404, that equity doesn’t flow anywhere. It’s wasted. Now imagine a popular blog post with 15 internal links, 3 of which point to deleted pages. Twenty percent of the equity that post could distribute to other pages is vanishing.

External links compound this. When another site links to your page with a typo in the URL, or links to a page you’ve since deleted, the link equity they intended to send you hits a 404 instead. Google’s John Mueller has confirmed that fixing these inbound broken links can recover meaningful value, especially when the linking page is authoritative.

Damage 3: User Experience and Trust

This is the hardest to quantify but the easiest to understand. A user who clicks a link and sees a 404 page loses trust in your site. They might not click your links again. They might leave entirely. On an e-commerce site, a broken link to a product page is a direct lost sale. On a content site, it’s a lost reader.

I tracked this on one of my projects. After fixing 12 broken internal links on a high-traffic guide page, average session duration increased by 18%. Bounce rate dropped 7 percentage points. The page wasn’t suddenly ranking higher, but visitors were staying longer because they could actually follow the references I pointed them to.

The Three Types of Broken Links

Before diving into detection methods, you need to understand that not all broken links are created equal. They come from different sources, require different detection approaches, and have different levels of urgency.

Type 1: Internal Broken Links. These are links on your own pages pointing to other pages on your site that no longer exist. You deleted a blog post, changed a URL structure, or made a typo in an href. These are the most common and the easiest to fix because you control both ends. A site crawl will find them all.

Type 2: Inbound Broken Links (External). These are links on other websites pointing to your site using a URL that no longer works. Maybe they linked to /blog/old-post and you renamed it to /blog/new-post without setting up a redirect. Or they made a typo. You can’t edit their link, but you can create a 301 redirect on your end to capture the equity. These show up in your 404 logs and server logs.

Type 3: External Outbound Broken Links. These are links on your pages pointing to external websites that have gone down, moved, or deleted the linked page. You linked to a great resource two years ago, and now it returns 404. This doesn’t waste your crawl budget as aggressively, but it degrades user experience and can hurt your site’s perceived quality. Google has stated that too many broken outbound links can be a negative quality signal.

Each type requires a different detection strategy. Let’s walk through them.

Method 1: Google Search Console Coverage Report

Google Search Console is your first line of defense for detecting broken internal links. Google has already crawled your site, found the broken URLs, and conveniently organized them in a report. This costs nothing and requires zero technical setup beyond GSC verification.

Finding 404 Errors in GSC

Navigate to Settings > Coverage in the old GSC interface, or Pages in the new Search Console. Look for the “Excluded” or “Not indexed” category, then filter for “404” or “Not found” error types.

You’ll see a list of URLs that Googlebot tried to crawl but received a 404 response. For each URL, GSC tells you:

• The exact URL that returned 404
• When Google last attempted to crawl it
• The number of referring pages, if available
• The discovery method (how Google found the URL)

The “referring pages” data is critical. It tells you where the broken link lives. Click any 404 URL in the report, and GSC shows you “Discovered from” or “Linked from” pages. These are pages on your site (or external sites) that contain links pointing to the broken URL.

What GSC Doesn’t Tell You

GSC’s coverage report has limitations you should understand:

It only shows URLs Google has discovered. If a broken link exists on a page that Google hasn’t crawled yet, it won’t appear. For new sites or recently published pages, there’s a lag of days or weeks before broken links show up.

Referring page data is incomplete. GSC shows a sample of referring pages, not all of them. I’ve seen cases where a broken URL was linked from 15 pages, but GSC only listed 3. Use it as a starting point, not a complete map.

It doesn’t categorize by severity. A 404 on a deleted blog post that nobody links to is harmless. A 404 on your most-linked-to page is critical. GSC lists them side by side without prioritization. You need to cross-reference with your analytics or internal link data to assess impact.

External outbound broken links are invisible. GSC tracks URLs on your domain that return errors. It doesn’t check whether the external links on your pages are still alive.

Building a GSC Workflow

Here’s how I use GSC for broken link detection:

1. Check the Pages report weekly. I look for any new “not found” URLs that appeared since the last check. GSC’s date filter helps here. I compare the current week’s 404 list to the previous week.

2. Export the full list monthly. Click “Export” in the Pages report and download the URL list as CSV. This gives you a baseline to track over time. If the number of 404s is growing, something in your site structure is generating broken URLs.

3. Cross-reference “Linked from” with your analytics. For each 404 URL, check the referring pages. If a high-traffic page is linking to a 404, fix it immediately. If a low-traffic archived page links to a 404, it’s lower priority.

4. Mark legitimate 404s as “no action needed.” If you intentionally deleted a page and want it to return 404 (which is the correct behavior for permanently removed content), leave it. Google will eventually stop crawling it. Don’t waste time redirecting every deleted page, especially ones with no inbound links.

Method 2: Screaming Frog SEO Spider (Free Tier)

Screaming Frog is the industry standard site crawler, and its free tier allows you to crawl up to 500 URLs. For most indie sites and small blogs, 500 URLs is more than enough. If your site is larger, you can crawl specific sections by configuring the spider to only follow certain URL patterns.

Setting Up the Crawl

Download Screaming Frog from their website. The free version requires no registration. Open the tool, enter your domain in the URL bar, and click “Start.”

The spider will crawl your site, following internal links just like Googlebot does. It checks every link on every page and records the HTTP status code. This catches broken internal links, redirect chains, and broken outbound links in a single pass.

After the crawl completes, click the “Response Codes” tab and filter by “Client Error (4xx).” This shows every link that returned a 400-level error, with 404 being the most common.

Reading the Report

For each broken link, Screaming Frog shows you:

• Address: The URL that returned the error
• Status Code: Usually 404, but also 403 (forbidden) or 410 (gone)
• Status Source: Whether the error came from your server or an external server
• Inlinks: The pages on your site that link to this broken URL
• Anchor Text: The text used in the link
• Links Through Redirect: Whether the link goes through a redirect before hitting the error

The “Inlinks” column is the money column. It tells you exactly which pages contain the broken link. You don’t need to hunt through your site. The report points you directly to the problem.

Finding Broken Outbound Links

To find broken external links, switch the filter to “External” in the Response Codes tab. Screaming Frog checks external links by default, but the free tier limits you to checking links within the first 500 internal URLs. For most sites, this covers the majority of outbound links since most external links live on your most prominent pages.

Limitations of the Free Tier

The 500-URL limit is the main constraint. If your site has 800 pages, the crawl will stop at 500, leaving 300 pages unchecked. Workarounds:

Crawl by section. Instead of crawling your entire domain, crawl /blog/ separately from /resources/. Each section likely has fewer than 500 pages. Run multiple crawls and combine the results.

Exclude irrelevant URLs. In Screaming Frog’s Configuration > Exclude section, add patterns for URLs you don’t need to check, like /*?replytocom= or /*?amp=. This reduces the URL count and lets the crawler focus on real pages.

Schedule regular crawls. Set a reminder to run a full crawl once a month. Broken links accumulate over time as you add, modify, and delete content. A monthly crawl catches issues before they compound.

Method 3: DIY Python Broken Link Crawler

This is the method I use most. After relying on free online tools for months, I got frustrated by their rate limits, captcha walls, and shallow crawl depth. So I wrote my own crawler. It runs on any machine with Python installed, crawls your entire site regardless of size, and outputs a clean CSV report.

The script uses two libraries: requests for HTTP and BeautifulSoup for HTML parsing. Both are free and install in seconds.

The Crawler Script

#!/usr/bin/env python3
"""
Broken Link Checker — crawls a website and reports all broken links.
Outputs: broken_links.csv with columns [source_page, broken_url, status_code, anchor_text, link_type]
"""

import requests
from bs4 import BeautifulSoup
from urllib.parse import urljoin, urlparse
import csv
import time
from collections import deque

BASE_URL = "https://yourdomain.com"
DOMAIN = urlparse(BASE_URL).netloc
HEADERS = {"User-Agent": "BrokenLinkChecker/1.0 (SEO Audit)"}
DELAY = 0.5  # seconds between requests (be polite)
MAX_PAGES = 1000  # safety limit

visited = set()
broken_links = []
queue = deque([BASE_URL])

def is_internal(url):
    return urlparse(url).netloc == DOMAIN

def check_url(url):
    """Check a URL and return its status code."""
    try:
        response = requests.head(url, headers=HEADERS, timeout=10, allow_redirects=True)
        # Some servers block HEAD requests; fall back to GET
        if response.status_code == 405:
            response = requests.get(url, headers=HEADERS, timeout=10, stream=True)
            response.close()
        return response.status_code
    except requests.RequestException as e:
        return str(e)[:80]

def extract_links(url, html):
    """Extract all links from a page."""
    soup = BeautifulSoup(html, "html.parser")
    links = []
    for a_tag in soup.find_all("a", href=True):
        href = a_tag["href"]
        # Skip anchors, javascript, mailto, tel
        if href.startswith("#") or href.startswith("javascript:") or href.startswith("mailto:") or href.startswith("tel:"):
            continue
        full_url = urljoin(url, href)
        full_url = full_url.split("#")[0]  # strip fragments
        anchor_text = a_tag.get_text(strip=True)[:100]
        links.append((full_url, anchor_text))
    return links

# Main crawl loop
while queue and len(visited) < MAX_PAGES:
    current_url = queue.popleft()

    if current_url in visited:
        continue
    visited.add(current_url)

    try:
        response = requests.get(current_url, headers=HEADERS, timeout=10)
    except requests.RequestException:
        continue

    if "text/html" not in response.headers.get("Content-Type", ""):
        continue

    links = extract_links(current_url, response.text)

    for link_url, anchor_text in links:
        # Check external links immediately
        if not is_internal(link_url):
            status = check_url(link_url)
            if status and (isinstance(status, int) and status >= 400 or isinstance(status, str)):
                broken_links.append([current_url, link_url, status, anchor_text, "external"])
            time.sleep(0.2)
        else:
            # Queue internal links for crawling
            clean_url = link_url.split("?")[0]  # strip query params for dedup
            if clean_url not in visited and clean_url not in queue:
                queue.append(clean_url)

    time.sleep(DELAY)
    print(f"Crawled {len(visited)}/{MAX_PAGES}: {current_url}")

# Write results
with open("broken_links.csv", "w", newline="") as f:
    writer = csv.writer(f)
    writer.writerow(["source_page", "broken_url", "status_code", "anchor_text", "link_type"])
    writer.writerows(broken_links)

print(f"\nDone. Crawled {len(visited)} pages, found {len(broken_links)} broken links.")
print("Results saved to broken_links.csv")

How It Works

The crawler starts at your base URL, extracts all links on the page, and queues internal links for further crawling. For each link it discovers, it checks the HTTP status code. If the status is 400 or above, the link is logged as broken.

The script distinguishes between internal and external links. Internal links are followed and crawled. External links are checked once (HEAD request, with GET fallback) and not followed. This means the crawl stays focused on your domain while still catching broken outbound links.

Customizing the Crawler

Adjust the delay. The default 0.5-second delay between requests is polite. If your server is fast and you want speed, reduce it to 0.2. If you’re crawling a shared host with limited resources, increase to 1.0. Never set it to zero. Hammering your own server can cause timeouts that look like broken links.

Add exclusions. If your site has faceted navigation or URL parameters that generate infinite URL combinations, add a filter:

SKIP_PATTERNS = ["?sort=", "?filter=", "?page=", "/feed/", "/wp-json/"]

def should_skip(url):
    return any(pattern in url for pattern in SKIP_PATTERNS)

Check for soft 404s. Some servers return a 200 OK status for non-existent pages (a “soft 404”). Add a content check:

SOFT_404_INDICATORS = ["page not found", "404 error", "no results found"]

def is_soft_404(response):
    text = response.text.lower()
    return any(indicator in text for indicator in SOFT_404_INDICATORS) and len(response.text) < 5000

Export to JSON for programmatic use. If you want to integrate the results into a dashboard or CI pipeline, swap the CSV writer for JSON output:

import json
with open("broken_links.json", "w") as f:
    json.dump(broken_links, f, indent=2)

Running It on a Schedule

I set up a cron job that runs the crawler weekly and emails me the results. Here’s the crontab entry:

0 9 * * 1 cd /path/to/crawler && python3 broken_link_checker.py && \
  [ -s broken_links.csv ] && \
  mail -s "Weekly Broken Link Report" me@example.com < broken_links.csv

This runs every Monday at 9 AM. If the CSV has any content (broken links found), it emails me the report. If no broken links are found, I get no email. Silence is golden.

Method 4: Server Log Analysis for Inbound Broken Links

This is the method Google’s John Mueller specifically recommended in a Reddit thread. It’s the most accurate way to find broken inbound links because it captures every request that hits your server, including requests from external sites linking to non-existent pages.

What Server Logs Show You

Your web server (Nginx, Apache, Cloudflare, or whatever you use) logs every HTTP request it receives. Each log entry includes:

• Timestamp: When the request was made
• Client IP: Who made the request
• Method: GET, POST, HEAD, etc.
• URL path: The requested path
• Status code: 200, 301, 404, 500, etc.
• Referer: The page that linked to this URL (if available)
• User agent: What made the request (browser, Googlebot, etc.)

The status code and referer fields are what you need. Filter for 404 status codes, and look at the referer to see where the broken link lives.

Extracting 404s from Nginx Logs

If you’re on Nginx (common on VPS setups), the default access log is at /var/log/nginx/access.log. Here’s a one-liner to extract all 404 errors with their referers:

awk '($9 ~ /404/)' /var/log/nginx/access.log | \
  awk '{print $7, $11}' | sort | uniq -c | sort -rn > 404_report.txt

This outputs each 404 URL with its referer, sorted by frequency. The most frequently hit 404 URLs are your highest-priority fixes because they’re getting the most traffic (and wasting the most equity).

Cloudflare Workers Logs

If you’re on Cloudflare Pages or Workers (which I use for most of my projects), you don’t have traditional server logs. But you can add lightweight logging to your worker:

// In your worker's fetch handler
export default {
  async fetch(request, env) {
    const url = new URL(request.url);
    const response = await handleRequest(request);

    if (response.status === 404) {
      const referer = request.headers.get("referer") || "direct";
      const ua = request.headers.get("user-agent") || "unknown";
      console.log(`404\t${url.pathname}\t${referer}\t${ua}`);
    }

    return response;
  }
};

This logs every 404 to Cloudflare’s dashboard under Workers > Logs. You can view real-time 404 hits and their referers. For persistent logging, pipe these to a KV store or D1 database.

Using GSC Instead of Logs

If you don’t have server log access (many shared hosting environments don’t provide it), Google Search Console is your fallback. The Pages report shows URLs Google found that returned 404. While this only captures URLs Google has crawled (not all incoming traffic), it covers the most important ones because Google prioritizes crawling URLs it discovers via links.

Method 5: Browser Extensions for Single-Page Audits

Sometimes you don’t need a full crawl. You’re editing a specific page and want to verify all its links work before publishing. Browser extensions are perfect for this.

Check My Links

Check My Links is a free Chrome extension that checks every link on the current page and highlights valid links (green) and broken links (red). It’s instantaneous and works on any page you’re viewing.

Use cases:

• Pre-publishing check. Before publishing a new blog post, load the draft in your browser and run Check My Links. It catches typos and broken references instantly.
• Post-edit verification. After updating an old post, run it to make sure you didn’t break any links during the edit.
• Competitor analysis. Run it on a competitor’s page to find broken outbound links you could potentially use for broken link building outreach.

Limitations

Browser extensions only check links on the current page. They don’t crawl your site. They’re a spot-check tool, not a comprehensive audit. Use them in conjunction with a full crawl, not instead of one.

How to Fix Each Type of Broken Link

Finding broken links is half the job. Fixing them correctly is the other half. The fix depends on the type of broken link.

Fixing Internal Broken Links

If the target page still exists but moved: Set up a 301 redirect from the old URL to the new URL. This preserves link equity and ensures any bookmarks or external links pointing to the old URL still work.

On Nginx:

location /blog/old-post/ {
    return 301 /blog/new-post/;
}

On Cloudflare Pages, add a _redirects file:

/blog/old-post/ /blog/new-post/ 301

If the target page was deleted and has no replacement: You have two options. If the page had meaningful inbound links or traffic, create a 301 redirect to the most relevant existing page. If nobody linked to it and it had no traffic, leave the 404. Google has confirmed that 404s for genuinely deleted content are correct behavior and won’t hurt your site.

If the link was a typo: Simply fix the href in the source page. This is the most common and easiest fix. Your crawl report tells you the exact source page and broken URL, so you know exactly what to correct.

Fixing Inbound Broken Links

You can’t edit the link on someone else’s site, but you can redirect the broken URL on your end:

1. Identify the broken inbound URL from your server logs or GSC.
2. Find the most relevant existing page on your site.
3. Create a 301 redirect from the broken URL to the relevant page.

This captures the link equity the external site intended to send you. The linking site doesn’t need to change anything.

If the broken inbound link is from a high-authority site and the redirect feels forced (the linked content genuinely doesn’t exist anymore), consider recreating a page at the old URL with relevant content instead of redirecting. This gives the linking site what they expected and preserves the contextual relevance of the link.

Fixing External Outbound Broken Links

For links on your pages pointing to dead external sites:

Find an alternative. If the linked resource has moved, update your link to the new URL. Use the Wayback Machine to find where the content went. Search for the title or key phrases to find a mirror.

Remove the link. If the resource is gone and no alternative exists, remove the link entirely or replace it with a brief note explaining the resource is no longer available.

Link to the Wayback Machine. If the resource was valuable and no replacement exists, consider linking to the archived version: https://web.archive.org/web/*/original-url. This preserves the reference for your readers.

The Hidden Opportunity: Broken Link Building

So far we’ve talked about broken links as a problem to fix. But there’s a flip side that most free-SEO guides skip entirely. Broken links on other people’s sites are one of the most effective free link building strategies available.

Here’s how it works. You find a broken outbound link on an authoritative site in your niche. You create content that replaces the dead resource. You reach out to the site owner, let them know about the broken link, and suggest your page as a replacement. If your content is genuinely useful, many site owners will update their link to point to you.

This strategy works because you’re solving a problem for the site owner, not asking for a favor. Nobody wants broken links on their pages. You’re doing them a service by pointing it out and offering a fix.

Finding Broken Links on Other Sites (Free)

Use the same tools and techniques from this guide, but point them at competitor or industry sites instead of your own:

1. Check My Links extension. Load any authoritative page in your niche that links out to resources. Run the extension. Any red links are broken outbound links on their site.

2. Screaming Frog crawl of a specific page. Enter a single URL (not a full domain) in Screaming Frog and set the crawl mode to “List.” This checks all links on that one page without crawling the entire site.

3. Python script for batch checking. Modify the DIY crawler to accept a list of external URLs instead of crawling a domain. Feed it the URLs of resource pages in your niche and let it find broken links across all of them.

The Outreach

Keep it short and genuine. Don’t use templates that smell like mass outreach. Here’s the format that works:

Subject: Broken link on your [topic] resource page

Hi [name],

I was reading your guide on [topic] and noticed the link to [dead resource] is no longer working. I’ve written a similar resource on [your topic] that covers [specific angle]. Would it be helpful if I sent you the link?

[your name]

That’s it. No paragraphs about how amazing your content is. No fake compliments. Just a helpful heads-up and a low-pressure offer. Response rates for broken link building outreach typically run 5-15%, which is high for cold outreach.

What Makes This Free

Broken link building requires no paid tools. You’re using browser extensions, free crawlers, and email. The “cost” is your time: finding broken links, creating replacement content, and sending outreach emails. But the ROI can be significant. A single successful placement on an authoritative site can be worth more than months of content creation in terms of ranking impact.

When NOT to Fix 404s

Not every 404 needs fixing. Google’s John Mueller has explicitly said this: “Keep in mind that you don’t have to fix 404 pages, having things go away is normal and fine.”

Here’s when you should leave a 404 alone:

Permanently deleted content with no inbound links. If you deleted an old blog post that nobody links to and that has no traffic, the 404 is correct. Google will stop crawling it after a few weeks. Creating a redirect wastes your time and can create redirect chains if you ever restructure again.

URL parameters that bots invent. Googlebot sometimes constructs URLs by modifying existing ones (appending parameters, testing variations). These “discovered” URLs return 404 and show up in GSC. They’re not real broken links. You don’t need to fix them. If they become a crawl budget problem, add the patterns to your robots.txt or use the URL Parameters tool in GSC.

Legitimate 410 Gone responses. If you intentionally deleted content and want to signal to Google that it’s permanently gone, return a 410 status code instead of 404. Google treats 410 as a stronger signal to stop crawling the URL sooner. You don’t need to fix or redirect these.

Temporary 404s during migration. If you’re in the middle of a site migration, some URLs may temporarily 404 while you set up redirects. Don’t panic. Fix redirects systematically, then use GSC’s “Validate Fix” button to ask Google to recrawl.

The key insight is this: fixing a 404 is only valuable when the broken URL receives traffic, has inbound links, or is linked from important pages on your site. If none of those are true, the 404 is harmless and fixing it is busywork.

Building a Broken Link Monitoring Workflow

Finding and fixing broken links shouldn’t be a one-time project. Links break constantly as you add content, restructure pages, and external sites change. Here’s the workflow I’ve settled on after months of iteration:

Weekly (5 Minutes)

1. Check GSC Pages report. Look for new 404 URLs since last week. If any appeared, check their “Linked from” data. Fix high-priority ones immediately.

2. Review server logs for 404 hits. If you have log access, scan the 404 report for any URLs getting significant traffic. These are often inbound broken links from external sites.

Monthly (30 Minutes)

1. Run a full site crawl. Use Screaming Frog (free tier) or the Python crawler. This catches internal broken links and broken outbound links that GSC doesn’t report.

2. Export and compare. Compare this month’s broken link report to last month’s. If the count is growing, investigate why. Common causes: URL structure changes without redirects, deleted pages without redirects, external sites going offline.

3. Fix all high-impact broken links. Cross-reference the crawl report with your analytics. Broken links on high-traffic pages get fixed first. Broken links from authoritative external sites get redirected second. Everything else can wait.

Quarterly (1 Hour)

1. Audit redirect chains. Over time, redirects accumulate and chain together (Page A redirects to Page B which redirects to Page C). Run a crawl that follows redirects and identifies chains longer than two hops. Collapse them by redirecting Page A directly to Page C.

2. Review robots.txt and sitemap. Make sure your sitemap doesn’t include URLs that return 404. Make sure robots.txt isn’t blocking important pages that have since been moved.

3. Check for orphaned redirects. If you redirected /old to /new two years ago and no links point to /old anymore, the redirect is unnecessary. You can remove it to simplify your server config. This is optional cleanup, not critical.

Common Broken Link Patterns to Watch For

After auditing multiple sites, I’ve noticed recurring patterns that generate broken links. Watching for these proactively prevents them:

URL structure changes. Every time you change your URL structure (e.g., switching from /blog/post-title to /blog/2024/post-title), every existing internal link to the old URL pattern breaks. Before making structural changes, export all your URLs, map old to new, and set up 301 redirects for every page.

Deleted content without redirects. You delete an old blog post but forget that three other posts link to it. Always search your site for internal links before deleting a page. In WordPress, this is easy with Link Whisper. On a static site, grep your content files for the URL.

External sites that reorganize. You linked to example.com/guide and they moved it to example.com/tutorials/guide. The 404 isn’t your fault, but it’s your problem. The monthly crawl catches these.

Case sensitivity. On Linux servers, /About and /about are different URLs. If you accidentally use the wrong case in an internal link, it works on your local Mac (case-insensitive) but breaks in production (case-sensitive). Standardize on lowercase URLs.

Trailing slashes. Your CMS generates links with trailing slashes (/blog/post/) but you manually typed links without them (/blog/post). Depending on your server config, this can cause 301 redirects (not broken but slow) or 404s (broken). Pick one convention and stick to it.

Tool Summary

Here’s every tool mentioned in this guide and what it costs:

• Google Search Console — Free. Shows URLs that returned 404 during Google’s crawl, with referring page data.
• Screaming Frog SEO Spider — Free tier: 500 URLs. Comprehensive site crawl that catches internal, external, and redirect chain issues.
• Python + requests + BeautifulSoup — Free. DIY crawler with no URL limits, customizable to your exact needs.
• Server access logs — Free (if your hosting provides them). The most accurate source of 404 data including referers and user agents.
• Cloudflare Workers console.log — Free. Lightweight 404 logging for serverless deployments.
• Check My Links (Chrome extension) — Free. Instant single-page link checking for pre-publish verification.
• Wayback Machine — Free. Find archived versions of dead external resources.

Total cost: $0. Total tools: 7. The Python crawler does 80% of the work for free with no limits. GSC handles Google’s perspective. Server logs (or Cloudflare logging) catch inbound broken links. Browser extensions handle quick spot checks.

The Bottom Line

Broken links are the silent killers of SEO. They don’t trigger alerts. They don’t show up in analytics dashboards. They just quietly waste crawl budget, leak link equity, and frustrate users until you go looking for them.

You don’t need Ahrefs at $99/month or Semrush at $130/month to find them. You need Google Search Console (which you should already have), a free site crawler, and optionally a Python script that runs on your schedule. The methods in this guide catch the same broken links that paid tools find, because the underlying data is the same. The difference is that paid tools package it in a nicer UI and charge you for the convenience.

The real work isn’t finding broken links. Any tool can do that. The real work is building a habit of checking regularly, prioritizing fixes by impact, and knowing when to leave a 404 alone. Run the crawl monthly. Fix the high-impact issues. Ignore the noise. That’s the entire job.