WordPress For Non-Techies: Free WordPress Tutorials – WPCompendium.org

How To Block WordPress From Search Engines

This tutorial is part of our WordPress configuration step-by-step tutorials, where we show you how to set up and configure your WordPress site or blog with no coding skills required.

***

How To Block Your WordPress Site From Search Engines

WordPress combines an easy-to-use content publishing and management system with a powerful business marketing tool that when expertly configured, allows search engines to easily find your site and quickly index your posts and pages. Most of our tutorials assume that you actually want search engines to index your site.

There may be instances, however, when you may want to block search engines from indexing your site.

This tutorial covers some of the possible reasons why you would want to block your site from being indexed by search engines and shows you how to prevent search engines from indexing your site using various methods, including configuring WordPress settings and using different WordPress features and plugins.

Why Block Your WordPress Site From Search Engines?

Some of the reasons why you may want to block search engines from indexing your WordPress site can include:

Blocking WordPress From Search Engines – Step-By-Step Tutorial

There are various methods you can use to prevent search engines from indexing your site.

These include:

How To Block WordPress From Search Engines – Configuring WordPress Settings

The first method you can use to help prevent search engines from crawling and indexing your site is to configure your WordPress Reading Settings

(You can discourage search engines from indexing your website)

After configuring your WordPress Reading Settings to discourage search engines from indexing your site, WordPress will also stop automatically notifying the update services listed in your WordPress Writing Settings

(You can block WordPress from automatically notifying update services)

To learn more about configuring your WordPress Writing Settings, see the tutorial below:

To learn more about configuring your WordPress Reading Settings, see the tutorial below:

Configuring the above settings will allow visitors continued access to your site and will instruct search engines not to index your site, but it cannot force-block access to your site. It is up to search engines to honor your request. Malware and spam bots will typically ignore it.

(Search Engines Discouraged)

robots.txt File

The next step in the process of blocking search engines from accessing your site is to configure a file in your server called robots.txt

(robots.txt)

A robots.txt file is used to instruct search engine spiders and co-operative bots which parts of your website they can or should not access.

To learn how to create, configure and add a robots.txt file to your server, see the tutorial below:

Blocking WordPress From Search Engines – Using Built-In WordPress Features

If you want to block or restrict certain parts of your website from being indexed by search engines, there are certain built-in features of WordPress you can use.

We have created step-by-step tutorials showing you how to block or restrict certain parts of your website from being indexed by search engines using all of the above built-in features of WordPress.

To learn more about protecting content in WordPress, see the tutorial below:

How To Block WordPress From Search Engines – Using WordPress Plugins

You can use WordPress plugins to prevent your site or restrict sections of your site or content from being indexed by search engines. Some of the strategies you can use with plugins include:

Blocking Content From Search Engines With Yoast SEO

If you have the Yoast SEO WordPress plugin installed, you can configure its settings to prevent search engines from indexing Posts, Pages, and Media on your site.

To access these settings, go to your WordPress admin menu and select SEO > Titles & Metas

(Yoast SEO – Titles & Metas)

When the Titles & Metas screen opens, click on the Post Types tab …

(Yoast SEO: Titles & Metas – Post Types)

In the Post Types section, enable no index in the Meta Robots options to stop search engines from indexing your Posts, Pages, and Media …

(Prevent Search Engines From Indexing Posts, Pages, And Media)

You can also prevent search engines from indexing individual posts or pages on your site by clicking on the plugin’s ‘Advanced’ tab below the Post or Page editor and selecting the following:

(Yoast SEO – Advanced Options)

You can also choose to noindex a certain category, tag, or custom taxonomy, by configuring the plugin’s options in that feature’s settings. For example, the screenshot below shows the WordPress SEO plugin’s settings when editing a category and how to prevent search engines from indexing that entire category …

(Yoast SEO – Post Category Settings)

So far, you have learned how to block search engine results by restricting indexing of your site, sections of your site, and individual posts or pages. Restricting indexing on your site, however, will not remove a page or post that was previously indexed by a search engine, at least until the next time the search engine bots attempt to spider your site (or perhaps even longer).

If a page of your site is already indexed and you would like it removed from the search engines, then do the following:

First, restrict indexing of the page or post using the methods described above. Once you have done this, you can request Google to remove an indexed site, page or post from its search database.

How To Remove Content That Has Already Been Indexed

Let’s say that you have published or created a post that Google has already found and indexed, and now you want it removed from Google’s search engine results.

Normally, when a page is updated or removed, it will eventually and automatically fall out of Google’s search results. You don’t need to do anything to make this happen. However, if you urgently need to remove content from Google’s search results, you can request expedited removal of those URLs.

For this example, let’s go through the process of asking Google to stop displaying content from a post that it has already indexed.

First, make sure you restrict indexing of the post that you would like removed from Google using one of the methods discussed earlier, then log into your Google Search Console

(Google Search Console Login)

To learn how to set up a Google Search Console account, see the tutorial below:

Once you have logged into your account, click on your site’s name. This will bring up a list of options for managing your domain …

(Search Console Dashboard)

Go to Search Console > Dashboard > Google Index

(Search Console > Dashboard > Google Index)

Inside the ‘Google Index’ menu section, click on Remove URLs

(Google Index > Remove URLs)

This brings you to the ‘Remove URLs’ screen.

Google will temporarily remove your URLs from its search results. Enter the relative URL of the post, page or directory you want Google to remove from its indexing database and click Continue …

(Remove URLs)

The relative URL is the part of the URL without the main domain name. So, in the example above:

Make sure you pay careful attention to the URL you are entering into this field because URLs are case sensitive.

Read the ‘More information’ section to learn more about removing content permanently.

Once you have specified the URL to be removed, the next step is to indicate the request type. From the drop-down menu, choose one of the following options:

(Remove URLs – Request Type)

Click Submit Request …

(Remove URLs – Submit Request)

Your request will be submitted.

For more information about using Google Search Console to block search engine access to your content or remove URLs from its search results, refer to the Search Console help pages below:

Hopefully, now you have a better understanding of methods that you can use to block your site and prevent search engines from accessing areas of your site and your content.

***

"I am beyond impressed with what you have put together. I can tell that you put a ton of hard work into building what you have. You have the absolute best content on WordPress I have ever seen!" - Robert T. Jillie

***

Recommended Video Courses For WordPress Users

How To Use cPanel

cPanel is a powerful and simple-to-use web hosting management software application that gives website owners the ability to quickly and easily manage their servers and websites using a simple and intuitive dashboard.

This video course will teach you how to use the main features of cPanel to manage your web hosting.

More info: How To Use cPanel

Recommended Video Courses For WordPress Users

How To Use Amazon S3

Learn how to set up and use Amazon S3 to upload, store, manage, and protect your site’s images, large media files, downloadable files, stream videos and more.

More info: How To Use Amazon S3

Recommended Video Courses For WordPress Users

How To Set Up A Forum On WordPress

Learn how to set up a forum for your visitors, prospects, customers, members, or affiliates using WordPress.

More info: How To Set Up A Forum On WordPress

Recommended Video Courses For WordPress Users

WordPress Security

Learn how to keep your WordPress site or blog secure and protected from malware, hackers and brute-force attacks.

More info: WordPress Security

Recommended Video Courses For WordPress Users

How To Set Up WordPress On Localhost

Learn how to install, set up, and locally host a fully functioning WordPress site on your computer.

More info: How To Set Up WordPress On Localhost

Recommended Video Courses For WordPress Users

Sell Online With PayPal

This video course shows you how to set up PayPal on your website to receive payments online, with additional tips on using PayPal payment tools.

More info: Sell Online With PayPal

Recommended Video Courses For WordPress Users

How To Install, Set Up & Use WordPress Multisite

Learn how to quickly and securely set up an online community of websites or blogs using the powerful WordPress Multisite feature.

More info: WordPress Multisite

Recommended Video Courses For WordPress Users

How To Use FTP

This video course shows you how to use FTP (File Transfer Protocol) to transfer and upload files between your hard drive and your server using a free FTP program called Filezilla.

More info: How To Use FTP

Recommended Video Courses For WordPress Users

Using Password Managers

Password Managers provide an easy and secure way to keep track of all your passwords. This video course shows you how to use two FREE powerful password management tools.

More info: Using Password Managers

Recommended Video Courses For WordPress Users

Build A WordPress Site ... Fast!

Need a new website or blog in a hurry?

This video course shows you how to quickly install and configure a WordPress website or blog on your own domain name and be up and running in no time!

More info: Build A WordPress Site ... Fast!