Skip to content
How to Deindex a Site from Google: Complete Guide
Indexing Guide
14 min read

How to Deindex a Site from Google: Complete Guide

Learn how to deindex a site or webpage from Google using noindex, Search Console Removals, HTTP 410/404, X-Robots-Tag, and password protection.

Author

InstantIndexer Team

Published

How to deindex a site from Google depends on whether you want to remove a single page, an entire website, or specific content from search results. The most reliable methods include adding a noindex directive, deleting pages and returning a 404 or 410 HTTP status, or protecting content with a password. If you need immediate removal, Google's Removals tool can temporarily hide URLs while you implement a permanent solution.

One important distinction: deindexing is not the same as blocking crawling. Google must generally be able to crawl a page to discover its noindex directive. Blocking it through robots.txt can prevent Google from seeing that instruction, leaving the URL eligible to appear in search results.

What Does Deindexing a Website from Google Mean?

Deindexing means removing a webpage, directory, subdomain, or entire website from Google's search index so that it no longer appears in normal Google Search results.

It doesn't necessarily mean deleting the website from the internet.

For example, consider a website with these pages:

  • `example.com/`
  • `example.com/blog/`
  • `example.com/about/`
  • `example.com/private/`
  • You might want to remove only the `/private/` section while keeping the homepage, blog, and about page visible in Google.

    Alternatively, you might want to remove every page because you're shutting down the website or preparing to launch a completely new version.

    Google supports different methods for these situations. Choosing the right one matters because some methods provide temporary removal, while others communicate a permanent indexing restriction.

    Deindexing vs. Removing a Website vs. Blocking Crawlers

    These terms are related, but they don't mean the same thing.

    ActionWhat Happens
    DeindexingA page or site is removed from Google Search results.
    Deleting a pageThe content is removed from your server.
    Blocking crawlingGooglebot is prevented from accessing a URL.
    noindexGoogle is instructed not to include a page in search results.
    Temporary removalGoogle hides a URL from search results for a limited period.

    For most website owners, the objective is to remove unwanted URLs from Google's index without accidentally blocking important pages.

    7 Ways to Deindex a Site from Google

    The following methods cover different situations, from removing a single URL to taking an entire website out of Google Search.

    1. Use the Google Search Console Removals Tool

    If your website is already indexed and you want to hide pages quickly, Google's Removals tool is usually the easiest starting point.

    It allows verified website owners to request temporary removal of URLs or URL prefixes from Google Search.

    Google states that successful requests can take effect within approximately one day. However, this is a temporary measure, not permanent deindexing.

    How to remove a URL using Search Console:

  • Open Google Search Console.
  • Select your verified website property.
  • Navigate to Indexing > Removals.
  • Open the Temporary Removals tab.
  • Click New Request.
  • Select the appropriate removal option.
  • Enter the URL or URL prefix you want to hide.
  • Submit the request.
  • There are two important removal options:

  • Temporarily remove URL: Hides a specific URL from Google Search.
  • Remove all URLs with this prefix: Hides URLs that begin with the specified URL prefix.
  • For example, if you want to hide an entire directory, you could submit:

    `https://example.com/private/`

    This is useful when you need to remove multiple pages within a specific section.

    Important limitation: Temporary removals last approximately six months (180 days). If the original content remains accessible and you haven't implemented a permanent solution, Google may show it again after the request expires. Use this tool for speed, then implement one of the permanent methods below.

    2. Add a Noindex Meta Tag to Your Website

    The noindex meta tag is one of the most reliable ways to tell Google not to include a webpage in search results.

    When Googlebot crawls a page containing this directive, it can process the instruction and remove that page from its index. Unlike deleting a page, this method allows the page to remain accessible to visitors who have its direct URL.

    How to add a noindex tag:

    Add the following HTML code inside the page's HTML head section:

    ``

    If you want search engines to avoid following links on the page as well, use:

    ``

    The first option prevents indexing. The second also provides a directive not to follow links on that page. For most deindexing situations, `noindex` alone is sufficient.

    How to deindex an entire website using noindex:

    If you want to remove your entire website from Google, you can add the directive to every indexable HTML page through a shared website template.

    Ensure the directive is present across all templates:

  • Homepage
  • Blog posts
  • Category pages
  • Product pages
  • Landing pages
  • Other indexable URLs
  • Critical rule: Don't block the affected URLs in `robots.txt` while expecting Google to read their noindex tags. Googlebot must be able to crawl the pages to discover the directives.

    3. Remove Pages Using HTTP 404 or 410 Status Codes

    If you no longer need a webpage, removing its content and returning the appropriate HTTP status code is a straightforward way to get it out of Google's index.

    HTTP StatusMeaningWhen to Use
    404 Not FoundThe requested page doesn't exist.A page has been removed or cannot be found.
    410 GoneThe page has been permanently removed.You intentionally deleted content and don't plan to restore it.

    Both status codes indicate that the requested content is unavailable. Google's documentation explains that previously indexed URLs returning 4xx responses are removed from the index over time.

    Workflow for deleted pages:

  • Remove the old page from your CMS or web server.
  • Configure the server to return a 404 or 410 response.
  • Remove the URL from your XML sitemap.
  • Update internal links pointing to the deleted page.
  • Check its HTTP response using a status-code checker or Search Console URL Inspection.
  • Avoid soft 404 errors:

    A common technical mistake is deleting page content but continuing to return HTTP 200 (for example, displaying "This product is no longer available" with a 200 OK header). Google may treat this as a soft 404. Always return a genuine 404 or 410 status code.

    4. Use the X-Robots-Tag HTTP Header

    The X-Robots-Tag is another way to tell Google not to index content. It works similarly to the noindex meta tag, but it is particularly useful for resources that don't contain HTML, such as:

  • PDF documents
  • Images
  • Videos
  • Downloadable spreadsheets and ZIP files
  • Configure your web server to return the response header:

    `X-Robots-Tag: noindex`

    Apache configuration:

    ``

    `Header set X-Robots-Tag "noindex"`

    ``

    NGINX configuration:

    `location ~* \.pdf$ { add_header X-Robots-Tag "noindex"; }`

    Choose X-Robots-Tag when controlling non-HTML resources or applying indexing rules globally at the web server layer.

    5. Password-Protect Your Website or Specific Pages

    If you want to prevent search engines and unauthorized visitors from accessing certain content entirely, password protection is the most secure option.

    This method is especially useful for:

  • Staging and development environments
  • Internal business portals
  • Client portals and dashboards
  • Confidential documents
  • When a page requires HTTP basic authentication or login credentials, Googlebot cannot access or parse its content, keeping it completely out of search results.

    6. Use a 301 Redirect When a Page Has Moved

    Sometimes you don't actually want to delete a page; you want to replace it with a newer, superior URL.

    In this situation, a permanent 301 redirect is the recommended SEO approach. A 301 redirect signals to search engines and visitors that the original URL has permanently migrated, passing ranking authority to the new target.

    Implementation guidelines:

  • Test the old URL to confirm it returns HTTP 301.
  • Verify it redirects directly to the destination without multi-hop redirect chains.
  • Update internal links and sitemaps to point directly to the destination URL.
  • Avoid redirecting unrelated deleted URLs to the homepage, as Google often flags non-relevant mass redirects as soft 404s.
  • 7. Use Google's Refresh Outdated Content Tool

    If you don't own the website hosting the content, or if content was deleted but Google continues displaying an outdated search snippet, use Google's public Refresh Outdated Content tool.

    ToolPurposeOwnership Required
    Search Console RemovalsTemporarily hide content from a site you own.Yes (verified property).
    Refresh Outdated ContentRequest update/removal of changed or deleted content.No.

    Open Google's Refresh Outdated Content tool, enter the public URL, and submit the request for Google to re-evaluate the cached snippet.

    How to Deindex an Entire Website from Google

    If your goal is to remove your complete website rather than individual URLs, choose one of these three strategies:

  • Keep site online but remove from Search: Implement site-wide `` across all page templates. Ensure robots.txt permits crawling so Googlebot reads the directive.
  • Permanently shut down the website: Remove the web files from the hosting server, return HTTP 410 Gone for all routes, and remove the domain's XML sitemap.
  • Emergency temporary hiding: Submit a URL-prefix removal request for `https://example.com/` in Google Search Console's Removals tool to hide all pages within 24 hours while applying permanent server-level configurations.
  • Noindex vs. Robots.txt vs. 404 vs. 301: Which Method Should You Choose?

    MethodMain PurposeBest Use Case
    NoindexPrevent search indexingKeep pages online but remove them from search.
    Robots.txtControl crawlingManage crawler access.
    404Indicate missing contentRemove deleted pages.
    410Indicate permanently removed contentPermanently delete pages.
    301 redirectMove a URL permanentlyReplace old URLs with relevant new pages.
    Password protectionRestrict accessKeep private content inaccessible to crawlers.
    Removals toolTemporarily hide search resultsUrgent removal requests.
    X-Robots-TagControl indexing through HTTP headersPDFs and other non-HTML files.

    Why You Shouldn't Use Robots.txt to Deindex a Website

    A robots.txt file controls crawl accessibility, not indexing status.

    If you add `Disallow: /page/` to robots.txt, Googlebot will stop crawling the page. However, if external websites link to that URL, Google can still index the URL and display it in search results without a snippet.

    Furthermore, if you block crawling on a page that contains a `noindex` tag, Googlebot can never crawl the page to see the `noindex` instruction, leaving the URL stuck in the index.

    Core rule:

  • To prevent indexing while allowing crawling: use `noindex`.
  • To delete content: return `404` or `410`.
  • To migrate content: return `301`.
  • To manage bot bandwidth: use `robots.txt`.
  • How to Check Whether Google Has Deindexed Your Website

    After adding noindex or deleting pages, verify Google's indexing state using these techniques:

    1. Use the Site Search Operator

    Query Google with `site:https://example.com/your-page/`. If Google returns "Your search did not match any documents", the URL is absent from the index.

    2. Use Search Console's URL Inspection Tool

    Inspect the URL in Google Search Console and check the reported status:

    StatusWhat It Means
    URL is on GoogleThe URL is currently eligible to appear in Google Search.
    URL is not on GoogleThe URL isn't currently indexed.
    Excluded by noindex tagGoogle detected a noindex directive.
    Not found (404)Google encountered a missing page.
    Page with redirectGoogle detected a redirect.
    Blocked by robots.txtGoogle cannot crawl the URL because of robots.txt restrictions.

    3. Review the Page Indexing Report

    Monitor your site-wide indexing health in Search Console under Indexing > Pages to track URLs shifting from "Indexed" to "Excluded by 'noindex' tag" or "Not found (404)".

    How Long Does It Take to Deindex a Website from Google?

    MethodExpected Timing
    Search Console RemovalsOften within 24 hours for successful requests
    Noindex meta tagAfter Googlebot recrawls and processes the page
    404 or 410After Googlebot recrawls and processes the URL
    Password protectionAfter Googlebot detects the 401/403 access restriction
    301 redirectAfter Google processes the redirect and canonical signals
    X-Robots-TagAfter Google crawls the resource and processes the header

    Common Mistakes to Avoid When Deindexing a Website

  • Blocking Googlebot before adding noindex: Blocking a URL in robots.txt prevents Googlebot from reading your noindex tag.
  • Relying solely on the Removals tool: Removals expire after ~180 days. Permanent directives must accompany temporary requests.
  • Adding noindex only to the homepage: Subpages, categories, and archive URLs remain indexed unless noindex is added sitewide.
  • Forgetting non-HTML resources: PDFs and images require `X-Robots-Tag: noindex` headers.
  • Redirecting all deleted pages to the homepage: Triggers soft 404 warnings and confuses search engines. Return a clean 404/410 instead.
  • Assuming sitemap removal deindexes URLs: Removing URLs from XML sitemaps only reduces discovery cues; it does not instruct Google to drop indexed URLs.
  • Can You Use InstantIndexer to Deindex URLs from Google?

    InstantIndexer is specialized for URL discovery, rapid search engine submissions, and indexing monitoring across Google and Bing.

    For deindexing, website owners should rely on official protocols:

  • `noindex` for pages kept online
  • `404` or `410` for permanently removed content
  • Password authentication for staging and confidential portals
  • Google Search Console Removals for immediate 24-hour temporary suppression
  • Google's Indexing API does support URL deletion notifications, but Google strictly limits Indexing API eligibility to pages with JobPosting or BroadcastEvent video schema. For standard websites, applying proper HTTP response codes and meta tags remains the industry standard.

    Frequently Asked Questions

    How do I completely remove my website from Google?

    Add a noindex meta tag to every page template and ensure robots.txt permits crawling. For non-HTML files, configure your server to return `X-Robots-Tag: noindex`. If urgent, submit a URL-prefix removal in Google Search Console to hide the entire domain within 24 hours.

    Does robots.txt remove a website from Google?

    No. Robots.txt restricts crawling, not indexing. Google can still index and display a URL if other websites link to it. Use noindex or authentication instead.

    Can I deindex a website without deleting it?

    Yes. Adding `` keeps your pages accessible to direct visitors while directing search engines to exclude them from search results.

    How can I remove a website from Google within 24 hours?

    Submit a temporary removal request via Google Search Console's Removals tool. Successful requests hide URLs from search results in approximately one day.

    Will deleting my website remove it from Google automatically?

    Yes, once Googlebot recrawls the URLs and receives HTTP 404 or 410 responses, it will gradually drop them from the index.

    What is the difference between noindex and nofollow?

    `noindex` tells search engines not to index the webpage. `nofollow` instructs search engines not to follow the outbound hyperlinks on that page.

    Can I deindex only one folder of my website?

    Yes. Apply noindex to pages in that specific directory (e.g., `/private/`) or submit a URL-prefix removal in Google Search Console for that path.

    Does Google automatically remove pages with a 404 error?

    Yes. As Googlebot recrawls URLs returning genuine HTTP 404 Not Found responses, it eliminates them from the search index.

    Can I remove my website from Google but keep it indexed on Bing?

    Yes. You can use search-engine-specific directives, such as ``, which targets only Google while allowing Bingbot to continue indexing.

    Will deindexing my website affect its SEO rankings?

    Yes. Deindexed pages cannot rank or receive organic traffic from Google Search as long as they remain outside the index.

    Ready to Speed Up Your Indexing?

    Get your pages indexed in 24-48 hours with InstantIndexer. Join 2,000+ SEO professionals using our automated API integration.

    Start Your Free Trial