WordPress For Non-Techies: Free WordPress Tutorials – WPCompendium.org

How To Block WordPress From Search Engines

This tutorial is part of our WordPress configuration tutorial series, where we show you how to set up and configure your WordPress site or blog with no coding skills required.

***

How To Block WordPress From Search Engines

WordPress combines an easy-to-use content publishing and management system with a powerful business marketing tool that when expertly configured, allows search engines to easily find your site and quickly index your posts and pages. Most of our tutorials assume that you actually want search engines to index your site.

There may be instances, however, when you may want to block search engines from indexing your site.

This tutorial covers some of the possible reasons why you would want to block your site from being indexed by search engines and shows you how to prevent search engines from indexing your site using various methods, including configuring WordPress settings and using different WordPress features and plugins.

Why Block Your WordPress Site From Search Engines?

Some of the reasons why you may want to block search engines from indexing your WordPress site can include:

How To Block WordPress From Search Engines – Tutorial

There are various methods you can use to prevent search engines from indexing your site.

These include:

How To Block Your WordPress Site From Search Engines – Configuring WordPress Settings

The first method you can use to help prevent search engines from crawling and indexing your site is to configure your WordPress Reading Settings

(You can discourage search engines from indexing your website)

After configuring your WordPress Reading Settings to discourage search engines from indexing your site, WordPress will also stop automatically notifying the update services listed in your WordPress Writing Settings

(You can block WordPress from automatically notifying update services)

To learn more about configuring your WordPress Writing Settings, see the tutorial below:

To learn more about configuring your WordPress Reading Settings, see the tutorial below:

Configuring the above settings will allow visitors continued access to your site and will instruct search engines not to index your site, but it cannot force-block access to your site. It is up to search engines to honor your request. Malware and spam bots will typically ignore it.

(Search Engines Discouraged)

robots.txt File

The next step in the process of blocking search engines from accessing your site is to configure a file in your server called robots.txt

(robots.txt)

A robots.txt file is used to instruct search engine spiders and co-operative bots which parts of your website they can or should not access.

To learn how to create, configure and add a robots.txt file to your server, see the tutorial below:

How To Block Your WordPress Site From Search Engines – Using Built-In WordPress Features

If you want to block or restrict certain parts of your website from being indexed by search engines, there are certain built-in features of WordPress you can use.

We have created step-by-step tutorials showing you how to block or restrict certain parts of your website from being indexed by search engines using all of the above built-in features of WordPress.

To learn more about protecting content in WordPress, see the tutorial below:

How To Block Your WordPress Site From Search Engines – Using WordPress Plugins

You can use WordPress plugins to prevent your site or restrict sections of your site or content from being indexed by search engines. Some of the strategies you can use with plugins include:

Blocking Content From Search Engines With Yoast SEO

If you have the Yoast SEO WordPress plugin installed, you can configure its settings to prevent search engines from indexing Posts, Pages, and Media on your site.

To access these settings, go to your WordPress admin menu and choose SEO > Titles & Metas

(Yoast SEO – Titles & Metas)

When the Titles & Metas screen opens, click on the Post Types tab …

(Yoast SEO: Titles & Metas – Post Types)

In the Post Types section, enable no index in the Meta Robots options to stop search engines from indexing your Posts, Pages, and Media …

(Prevent Search Engines From Indexing Posts, Pages, And Media)

You can also prevent search engines from indexing individual posts or pages on your site by clicking on the plugin’s ‘Advanced’ tab below the Post or Page editor and selecting the following:

(Yoast SEO – Advanced Options)

You can also choose to noindex a certain category, tag, or custom taxonomy, by configuring the plugin’s options in that feature’s settings. For example, the screenshot below shows the WordPress SEO plugin’s settings when editing a category and how to prevent search engines from indexing that entire category …

(Yoast SEO – Post Category Settings)

So far, you have learned how to block search engine results by restricting indexing of your site, sections of your site, and individual posts or pages. Restricting indexing on your site, however, will not remove a page or post that was previously indexed by a search engine, at least until the next time the search engine bots attempt to spider your site (or perhaps even longer).

If a page of your site is already indexed and you would like it removed from the search engines, then do the following:

First, restrict indexing of the page or post using the methods described above. Once you have done this, you can request Google to remove an indexed site, page or post from its search database.

How To Remove Content That Has Already Been Indexed

Let’s say that you have published or created a post that Google has already found and indexed, and now you want it removed from Google’s search engine results.

Normally, when a page is updated or removed, it will eventually and automatically fall out of Google’s search results. You don’t need to do anything to make this happen. However, if you urgently need to remove content from Google’s search results, you can request expedited removal of those URLs.

For this example, let’s go through the process of asking Google to stop displaying content from a post that it has already indexed.

First, make sure you restrict indexing of the post that you would like removed from Google using one of the methods discussed earlier, then log into your Google Search Console

(Google Search Console Login)

To learn how to set up a Google Search Console account, see the tutorial below:

Once you have logged into your account, click on your site’s name. This will bring up a list of options for managing your domain …

(Search Console Dashboard)

Go to Search Console > Dashboard > Google Index

(Search Console > Dashboard > Google Index)

Inside the ‘Google Index’ menu section, click on Remove URLs

(Google Index > Remove URLs)

This brings you to the ‘Remove URLs’ screen.

Google will temporarily remove your URLs from its search results. Enter the relative URL of the post, page or directory you want Google to remove from its indexing database and click Continue …

(Remove URLs)

The relative URL is the part of the URL without the main domain name. So, in the example above:

Make sure you pay careful attention to the URL you are entering into this field because URLs are case sensitive.

Read the ‘More information’ section to learn more about removing content permanently.

Once you have specified the URL to be removed, the next step is to indicate the request type. From the drop-down menu, choose one of the following options:

(Remove URLs – Request Type)

Click Submit Request …

(Remove URLs – Submit Request)

Your request will be submitted.

For more information about using Google Search Console to block search engine access to your content or remove URLs from its search results, refer to the Search Console help pages below:

Hopefully, now you have a better understanding of methods that you can use to block your site and prevent search engines from accessing areas of your site and your content.

***

"I am beyond impressed with what you have put together. I can tell that you put a ton of hard work into building what you have. You have the absolute best content on WordPress I have ever seen!" - Robert T. Jillie

***