Table of contents
Quick answerWhy would you discourage indexing?WordPress featurenoindex meta tagrobots.txt filePassword protectionPluginTemporary RemovalsBonus tipRemove a page that’s been indexedFAQsGetting startedAdditional resources
Get the industry’s fastest hosting for WordPress◦ 100% network uptime
◦ Comprehensive security
◦ 24/7 support
Explore options

WordPress Guide → SEO → Discourage Search Engines Indexing Site

Discourage search engines from indexing this site: Why and how

Don’t want your site showing up in Google search results? Whether you’re still building your WordPress site or working on something private, there are simple ways to keep search engines out.

Let’s break down exactly how to discourage indexing, and why you might want to.

Quick answer: How to discourage search engines

The fastest way to block search engines in WordPress is to go to Settings > Reading > Search engine visibility and check the box for Discourage search engines from indexing this site. It adds a special rule to your site’s code (via robots.txt), to tell search engines to skip indexing your site. 

Note that this setting doesn’t guarantee full privacy—so if you need stronger protection, keep reading.

What is “Discourage Search Engines from Indexing This Site” on WordPress?

Indexing is the process search engines use to discover and store your website’s pages in their database. Once a page is indexed, it can show up in search results when someone enters a related query on Google, Bing, or other search engines.

The “Discourage Search Engines from Indexing This Site” setting in WordPress tells search engines not to include your site in their index. When you check the box, WordPress adds a line to your site’s robots.txt file:

User-agent: *
Disallow: /

That line politely tells all bots to avoid crawling or indexing your content. It’s not a lock on the door—it’s more like a “please don’t enter” sign. Most search engines follow this request, but it’s not enforced by WordPress.

Also important: If your site is already indexed, checking this box won’t remove it from search results. It only prevents new pages from being crawled.

Why would you discourage indexing?

There are several good reasons to keep search engines away—at least temporarily:

  • You’re building or redesigning your site. You don’t want visitors stumbling across half-finished pages or broken layouts in Google.
  • You’re using a staging or development site. Test environments should stay private. Discouraging indexing prevents search engines from picking up duplicate or irrelevant versions of your site.
  • You’re running a private site. Internal documentation, employee portals, client dashboards, or personal blogs shouldn’t be exposed in search results.
  • You’re creating gated or premium content. Membership content or paywalled courses should stay off Google unless you’re specifically optimizing teasers for SEO.
  • You want to test or draft content before going live. Sometimes, you just want to experiment or preview content without indexing it right away.

How to use the WordPress feature

To enable the built-in setting:

  • Log in to your WordPress dashboard
  • Go to Settings > Reading
  • Find the checkbox labeled Search engine visibility
  • Check the box: Discourage search engines from indexing this site
  • Click Save Changes

That’s it. WordPress updates your virtual robots.txt file automatically.

How to use the ‘noindex’ meta tag

The noindex meta tag tells search engines to skip indexing a specific page, even if the rest of your site is visible. It’s a precise way to hide select content from search results without blocking the entire site.

To add a noindex tag manually:

  • 1. Open your theme’s header.php file (or use a custom template)
  • 2. Insert this line inside the <head> section:
    • <meta name=”robots” content=”noindex, nofollow”>
  • 3. Save the file and clear your cache if needed

How to discourage indexing with the robots.txt file

If you want full control, you can create or modify a robots.txt file yourself.

  • 1. In the root directory of your site (usually via FTP or File Manager), create a file called robots.txt
  • 2. Add the following lines:
    • User-agent: *
    • Disallow: /
  • 3. Save and upload the file to your site’s root (e.g., https://example.com/robots.txt)
    • This works on any type of site and gives you more flexibility if you want to allow certain bots or block only specific folders.

Remember: Search engines can ignore this file. It’s a guideline, not a restriction.

How to discourage indexing with password protection

Password protection adds a harder barrier. If crawlers can’t access a page, they can’t index it.

Here are two simple ways to do this:

  • Password-protect your entire site via your hosting provider: Many hosts let you set up HTTP authentication or directory protection in your control panel.
  • Use a plugin to lock content: Plugins like Password Protected let you set a single password for your whole WordPress site. It’s a quick way to keep everything private during development.

Password protection blocks bots, users, and anyone who doesn’t have access credentials.

How to discourage indexing with a plugin

Several SEO and privacy plugins make it easy to manage indexing settings.

Popular options include:

  • Yoast SEO: Go to SEO > Settings > Site features > Site visibility, and toggle the option to discourage search engines.
  • All in One SEO: Navigate to Search Appearance > Advanced and turn on the “Discourage search engines” feature.
  • Coming Soon & Maintenance Mode Page & Under Construction: Not only discourages indexing but also displays a custom message to visitors while you work behind the scenes.

Plugins offer more granular control—like noindex settings per page, sitemaps, and canonical tags.

How to use Temporary Removals to discourage indexing

If a page is already indexed and you need to hide it quickly, Google Search Console’s Temporary Removals tool is your friend.

  • Go to Google Search Console
  • Choose your site property
  • Open the Removals section
  • Click New Request
  • Choose Temporarily remove URL, and enter the page you want to block

This removes it from search results for about six months. In the meantime, update the page to include a noindex meta tag or restrict access entirely.

Don’t let AI break your WordPress site

We like Google’s AI Overviews and other LLM summaries as much as the next DIY WordPress user. But when they cite inaccurate or outdated content, your site suffers.

Solution: Take control of your search results by adding Liquid Web as a preferred source. You’ll see our guides more readily when you search for help with your site.

Bonus tip: Use a staging site for WordPress development

If you’re making major changes to your WordPress site, don’t do it live.

Instead, use a staging environment. Managed hosts like Liquid Web include one-click staging so you can build, test, and revise without affecting your live site—or risking accidental indexing.

You can also set up staging manually using a subdomain (like staging.example.com) and discourage indexing via robots.txt or password protection.

How to remove a page that’s been indexed

Already have something in search results you want gone? Follow these steps:

  • Block it from future indexing
    Use a noindex meta tag or add it to your robots.txt file.
  • Request removal from Google
    Use the Google Search Console Removals tool
  • Clear caches and submit updated sitemaps
    This speeds up the re-crawl process and helps search engines process your changes.

You may also want to use the Remove URLs tool for immediate action.

Search engine indexing FAQs

Search engine indexing stores your website’s pages in a search engine’s database so they can appear in search results. Without indexing, search engines can still crawl your site, but they will not show those pages to users.

The three most common types are full indexing, incremental indexing, and selective indexing. Full indexing scans and stores all available pages, incremental indexing updates only changed content, and selective indexing focuses on pages a search engine considers most important.

A search engine indexer is the system that processes crawled pages and adds them to a search engine’s index. It analyzes content, structure, links, and metadata to decide how and when a page should appear in search results.

Turning off indexing tells search engines not to include your site or specific pages in search results. Your site will still work for visitors with a direct link, but organic traffic from search engines will stop until indexing is turned back on.

Additional resources

Comprehensive guide to securing WordPress with ModSecurity
→

This guide provides a comprehensive overview of how to use ModSecurity to enhance the security of your WordPress site.

Optimizing Largest Contentful Paint (LCP) in WordPress →

Improve Largest Contentful Paint for better performance and Core Web Vitals scores.

Why security matters for WordPress enterprise hosting
→

Use the blog as your guide to attacks to watch out for, security best practices, and steps to improve the WordPress protection you already have.

Originally from Albuquerque, New Mexico, Leslie Bowman has hopped around the country since graduating from undergrad. She focused on English and Latin American History in college and eventually attended graduate school in New York City. There, she developed a passion for short, persuasive arguments. Bowman found a love of copywriting and is now a Senior Copywriter at Liquid Web.