◦ Comprehensive security
◦ 24/7 support
WordPress Guide → SEO → Discourage Search Engines Indexing Site
Discourage search engines from indexing this site: Why & how
Don’t want your site showing up in Google search results? Whether you’re still building your WordPress site or working on something private, there are simple ways to keep search engines out.
Let’s break down exactly how to discourage indexing—and why you might want to.
Quick answer: How to discourage search engines
The fastest way to block search engines in WordPress is to go to Settings > Reading > Search engine visibility and check the box for Discourage search engines from indexing this site. It adds a special rule to your site’s code (via robots.txt), to tell search engines to skip indexing your site.
Note that this setting doesn’t guarantee full privacy—so if you need stronger protection, keep reading.
What is “Discourage Search Engines from Indexing This Site” on WordPress?
Indexing is the process search engines use to discover and store your website’s pages in their database. Once a page is indexed, it can show up in search results when someone enters a related query on Google, Bing, or other search engines.
The “Discourage Search Engines from Indexing This Site” setting in WordPress tells search engines not to include your site in their index. When you check the box, WordPress adds a line to your site’s robots.txt file:
User-agent: *
Disallow: /
That line politely tells all bots to avoid crawling or indexing your content. It’s not a lock on the door—it’s more like a “please don’t enter” sign. Most search engines follow this request, but it’s not enforced by WordPress.
Also important: If your site is already indexed, checking this box won’t remove it from search results. It only prevents new pages from being crawled.
Why would you discourage indexing?
There are several good reasons to keep search engines away—at least temporarily:
- You’re building or redesigning your site. You don’t want visitors stumbling across half-finished pages or broken layouts in Google.
- You’re using a staging or development site. Test environments should stay private. Discouraging indexing prevents search engines from picking up duplicate or irrelevant versions of your site.
- You’re running a private site. Internal documentation, employee portals, client dashboards, or personal blogs shouldn’t be exposed in search results.
- You’re creating gated or premium content. Membership content or paywalled courses should stay off Google unless you’re specifically optimizing teasers for SEO.
- You want to test or draft content before going live. Sometimes, you just want to experiment or preview content without indexing it right away.
How to use the WordPress feature
To enable the built-in setting:
- Log in to your WordPress dashboard
- Go to Settings > Reading
- Find the checkbox labeled Search engine visibility
- Check the box: Discourage search engines from indexing this site
- Click Save Changes
That’s it. WordPress updates your virtual robots.txt file automatically.
How to use the ‘noindex’ meta tag
The noindex meta tag tells search engines to skip indexing a specific page, even if the rest of your site is visible. It’s a precise way to hide select content from search results without blocking the entire site.
To add a noindex tag manually:
- 1. Open your theme’s header.php file (or use a custom template)
- 2. Insert this line inside the <head> section:
- <meta name=”robots” content=”noindex, nofollow”>
- 3. Save the file and clear your cache if needed
How to discourage indexing with the robots.txt file
If you want full control, you can create or modify a robots.txt file yourself.
- 1. In the root directory of your site (usually via FTP or File Manager), create a file called robots.txt
- 2. Add the following lines:
- User-agent: *
- Disallow: /
- 3. Save and upload the file to your site’s root (e.g., https://example.com/robots.txt)
- This works on any type of site and gives you more flexibility if you want to allow certain bots or block only specific folders.
Remember: Search engines can ignore this file. It’s a guideline, not a restriction.
How to discourage indexing with password protection
Password protection adds a harder barrier. If crawlers can’t access a page, they can’t index it.
Here are two simple ways to do this:
- Password-protect your entire site via your hosting provider: Many hosts let you set up HTTP authentication or directory protection in your control panel.
- Use a plugin to lock content: Plugins like Password Protected let you set a single password for your whole WordPress site. It’s a quick way to keep everything private during development.
Password protection blocks bots, users, and anyone who doesn’t have access credentials.
How to discourage indexing with a plugin
Several SEO and privacy plugins make it easy to manage indexing settings.
Popular options include:
- Yoast SEO: Go to SEO > Settings > Site features > Site visibility, and toggle the option to discourage search engines.
- All in One SEO: Navigate to Search Appearance > Advanced and turn on the “Discourage search engines” feature.
- Coming Soon & Maintenance Mode Page & Under Construction: Not only discourages indexing but also displays a custom message to visitors while you work behind the scenes.
Plugins offer more granular control—like noindex settings per page, sitemaps, and canonical tags.
How to use Temporary Removals to discourage indexing
If a page is already indexed and you need to hide it quickly, Google Search Console’s Temporary Removals tool is your friend.
- Go to Google Search Console
- Choose your site property
- Open the Removals section
- Click New Request
- Choose Temporarily remove URL, and enter the page you want to block
This removes it from search results for about six months. In the meantime, update the page to include a noindex meta tag or restrict access entirely.
Bonus tip: Use a staging site for WordPress development
If you’re making major changes to your WordPress site, don’t do it live.
Instead, use a staging environment. Managed hosts like Liquid Web include one-click staging so you can build, test, and revise without affecting your live site—or risking accidental indexing.
You can also set up staging manually using a subdomain (like staging.example.com) and discourage indexing via robots.txt or password protection.
How to remove a page that’s been indexed
Already have something in search results you want gone? Follow these steps:
- Block it from future indexing
Use a noindex meta tag or add it to your robots.txt file. - Request removal from Google
Use the Google Search Console Removals tool - Clear caches and submit updated sitemaps
This speeds up the re-crawl process and helps search engines process your changes.
You may also want to use the Remove URLs tool for immediate action.
Start optimizing your WordPress experience
Knowing how to control indexing gives you the power to manage visibility, protect privacy, and avoid SEO mistakes.
Ready to upgrade your WordPress experience in general? Professional hosting improves speeds, security, and reliability for a website and a brand that people find engaging and trustworthy. Liquid Web’s WordPress hosting options configure business-class servers and support plans specifically for WordPress websites.
Don’t want to deal with server management and maintenance? Our fully managed hosting for WordPress is the best in the industry. Our team are not only server IT experts, but WordPress hosting experts as well. Your server couldn’t be in better hands.
Click through below to explore all of our hosting for WordPress options, or chat with a WordPress expert right now to get answers and advice.
Additional resources
Comprehensive guide to securing WordPress with ModSecurity
→
This guide provides a comprehensive overview of how to use ModSecurity to enhance the security of your WordPress site.
Optimizing Largest Contentful Paint (LCP) in WordPress →
Improve Largest Contentful Paint for better performance and Core Web Vitals scores.
Why security matters for WordPress enterprise hosting
→
Use the blog as your guide to attacks to watch out for, security best practices, and steps to improve the WordPress protection you already have.
Originally from Albuquerque, New Mexico, Leslie Bowman has hopped around the country since graduating from undergrad. She focused on English and Latin American History in college and eventually attended graduate school in New York City. There, she developed a passion for short, persuasive arguments. Bowman found a love of copywriting and is now a Senior Copywriter at Liquid Web.