Cultured Digital logo Cultured Digital Talk to an SEO specialist

Learn · Technical SEO · Beginner

How to tell a real broken link from a 403 or 429

Learn to sort a broken-link report by status code, confirm by hand which links are really dead, and check that your own site is not refusing Google's crawler.

By Dean Cruddace · 45 minutes to do · Updated · Last reviewed

What 403 and 429 responses mean

A broken-link checker visits each link and reports the HTTP status code the server returned. Two codes cause many false alarms. A 403 means the server understood the request but refuses to fulfil it. A 429 means the sender has sent too many requests in a given time, which is called rate limiting. In both cases the server answered, so neither on its own says the page is gone.

This guide shows you how to sort a report by code, check flagged links by hand, and make sure your own site is not turning away Google's crawler by accident.

Why false broken links matter

Acting on a wrong label has costs. Removing a working link because a report called it broken thins out the sources your pages cite. More seriously, the same codes matter on your own site: Google says all 4xx errors except 429 are treated the same, meaning the content is treated as not existing and an indexed URL is removed from the index, while 429 and 5xx responses make its crawlers slow down. See how HTTP status codes affect Google's crawlers.

What you need to tell a real broken link from a 403 or 429

  • A broken-link report that shows the status code for each link
  • A web browser where you are logged out of the sites you are checking
  • Access to Google Search Console for your own site
  • Optionally, access to your server or firewall logs (your host can tell you where they are)

Tell a real broken link from a 403 or 429, step by step

  1. 1

    Export the flagged links with their codes

    From your checker, export every flagged link with its status code and the page it sits on. The code is what you need next.

    You will know it worked when You have a list with three columns: page, destination and status code.

  2. 2

    Sort the list by what each code means

    Group the links. 404 and 410 mean the content was not found or is gone (Google treats 4xx content as non-existent). 301 and 308 are permanent moves; Google follows them to the new address. 403 and 429 mean the server answered but refused or limited the request. Timeouts, refused connections and certificate errors are a different group again. See Google's status code table, RFC 9110 for 403 and RFC 6585 for 429.

    You will know it worked when Every flagged link sits in one group: gone, moved, refused or limited, or could not connect.

  3. 3

    Open the refused and limited links yourself

    Paste each 403 and 429 destination into a normal browser window, logged out. If the page loads for you, the link is not broken and the checker was refused. Leave the link in place. A 429 in particular is the server limiting how fast you ask, so slow down and try again later.

    You will know it worked when Each 403 or 429 link is marked as working or as really dead, based on what you saw.

  4. 4

    Retest connection failures later

    For timeouts, refused connections and certificate errors, try again at another time and from another network before deciding anything. A single failed visit is not proof.

    You will know it worked when Links that fail on every attempt are marked as down; the others are cleared.

  5. 5

    Fix the links that are really broken

    Replace or remove links that are genuinely gone. Update permanently moved links to the new address. Leave verified 403 and 429 links alone.

    You will know it worked when Re-run the checker: the only flags left are ones you have verified by hand.

  6. 6

    Check what Google receives from your own site

    If your site uses a firewall, CDN or bot protection, open Search Console and paste an important page address into the inspection bar. Click View crawled page to see the HTTP response and the returned HTML, and check it is your page and not an error. See the URL Inspection help.

    You will know it worked when The returned HTML is your real page content, and the response is a success.

  7. 7

    Watch your crawl responses

    Go to Settings, then Crawl stats, and read Host status and the Crawl responses table. Most responses should be 200 or other good codes, and Google says server errors (5xx) cause availability warnings and should be fixed if possible. Check again after any change to your firewall or CDN rules. See the Crawl Stats report help.

    You will know it worked when Host status is green and the share of error responses is low and not rising after changes.

  8. 8

    Allow the real Googlebot, not impostors

    If you do find Google's requests being refused, do not simply open the door to anything that claims to be Googlebot. Google explains how to verify a request: run a reverse DNS lookup on the IP address in your logs, check the domain is googlebot.com, google.com or googleusercontent.com, then run a forward lookup and confirm it returns the same IP. See verifying Google crawlers.

    You will know it worked when Rules allow verified Google requests, and the same pages now inspect cleanly in Search Console.

Common mistakes when you tell a real broken link from a 403 or 429

  • Deleting a link or page because a tool said "broken" without opening it in a browser.
  • Reading a 403 as "not found". It means the server refused the request, not that the page is absent.
  • Using 401 or 403 to limit how fast Google crawls. Google says not to; those codes have no effect on crawl rate.

Terms used when you tell a real broken link from a 403 or 429

HTTP status code
A number a server sends back to say how it handled a request, such as 200, 404 or 429.
403 Forbidden
The server understood the request but refuses to fulfil it.
429 Too Many Requests
The sender has sent too many requests in a given time; a form of rate limiting.
Rate limiting
A server or firewall rule that restricts how many requests one visitor may make in a period.
Googlebot
Google's main web crawler, the program that fetches pages for Google Search.

Go deeper

Technical SEOWhen a working link is reported as broken: 403s, 429s and bot protectionBroken-link reports are only as good as their classification. Two well-known sites were reported as dead while working for every human visitor, and the same behaviour…Read the in-depth article → Crawling and indexingHow crawling, robots.txt, sitemaps, canonicals and logs decide which pages Google can reach and index, explained from Google’s own documentation.See the service guide → Next in the pathHow to check a WordPress site before launch for SEORun a short set of checks on a new or rebuilt WordPress site before and just after it goes live, so nothing…Open the guide →
A crawl full of 403 and 429 errors you cannot make sense of?

Send Dean the crawler used, the sample addresses and the response codes. Telling bot protection from real faults is part of crawling and indexing work.

Talk to an SEO specialistTechnical SEO: crawling and indexing →