If you are into blogging or a website owner, you must be using Google Analytics as a tool for searching the traffic that you may be getting. The site traffic is what would play a significant role in improving the performance of your website. In fact, it should be one of the best options to check market trends and behavioral shifts of your audience. But, it remains the fact that even a powerful tool like Google Analytics is not free from bot traffic. How would you be able to detect, segregate and exclude bot traffic in Google Analytics? That is precisely what we would be learning in today’s post.

Exclude Bot Traffic in Google Analytics

Google Analytics is undoubtedly the best tool to analyze and understand the traffic on your site. Data plays a vital role in streamlining your reports, and thus it should be essential to keep your Google Analytics reports as much cleaner and accurate as possible.How would you be able to do it? Well, removing the bot traffic from your reports. There are several ways you can employ to detect and remove bot traffic quite quickly. But, before that, it would be a good idea to check out what exactly Bot Traffic is and how it affects you.

Bot Traffic – An Overview

Bot in the word Bot traffic refers to Robot. In essence, any traffic that does not originate from a human is termed as the Bot Traffic. By definition, we may define the Bot Traffic as the traffic generated by the nonhuman origin through robots and spiders.The Bots, Spiders, and Crawlers are a few software tools used on the internet to run some automated tasks on the internet. Bot traffic would appear as the legitimate traffic in many ways, but most of the times it would represent the low-quality traffic and spam most of the times. It should be noted that more than half of the traffic generated on a site comes under the Bot Traffic category.Bots can either be Ghost Bots and Zombie Bots. The Ghost Bots do not visit your site permanently but show up as referral traffic under your Google Analytics report. The Zombie Bots are riskier enough and can render your site entirely. This can, in fact, be the possible reason for producing analytics spam. Please note that your site also receives the Good Bots in the form of Search Engine crawlers and bots that are much needed and do not contribute to any spam or nonhuman activities on your site traffic.

Why Should Bot Traffic Be Removed?

Well, for obvious reasons. Just as we already mentioned, the Bot Traffic tends to spam and quite low in quality. It would provide a skewed representation of your data traffic. Given the fact that you have one Bot hit to your site for every human hit. This can adversely affect the performance and analysis of your site.When you are making an attempt at understanding your traffic and make decisions based on this data, it would be quite essential to have the right data before you. Data plays an indispensable role in your online marketing and other business decisions. When your Google Analytics report picks up Bot Traffic, it will result in skewed results and thus would lead to wrong assumptions and judgments. This can have detrimental effects on the performance of your site. In fact, it may even go to the extent of ruining your business as well. You need to ensure that your data is appropriate, accurate and free of errors.Having understood that, let us now move to find how to detect the Bot Traffic.

How to detect Bot Traffic?

We assume you are aware that the Bots do not trigger any JavaScript. This is the reason they are not shown in Google Analytics reports. That was precisely why Google Analytics reports were devoid of any Bot Traffic. However, the things have changed now. The Bot Traffic has begun ending up in the Google Analytics report and has been a concern for many website owners.How can you detect Bot Traffic then? There are a few indications that may give out the fact. In case, you have the habit of checking your Google Analytics reports on a daily basis; you can detect a sudden surge in traffic. This can be a clear indication of Bot Traffic. It should not be forgotten that there are some useful & essential Bots, like the Search Engine crawlers which generally do not interfere with your data.There are a few other indications to prove the existence of Bot traffic. Some of them are –

• Evidence of high value for new visitors.
• An enhanced bounce rate
• Unusually low session duration.

You can find the Bot traffic by checking out a few areas on Google Analytics. The following tips may be helpful.Browser and Operating System – Find if any specific browser and OS have been responsible for the sudden hike in traffic. That way, you would be able to narrow your analysis to the particular browser or OS. You may cross check it with the bounce rate and session duration if needed.Returning and New Users –Bots generally get picked up as new users but tend to offer you a low quality of engagement with the site. If you observe something like that, it would be practical enough to assume a high degree of bot traffic.Apply Custom Filters – You can find the domain that is generating a substantial amount of Bot traffic. Check the Network report on your Google Analytics and check out the dimension network domain. Once you can detect the network domain that has been causing the high surge in Bot traffic, you may exclude the traffic from the particular web-domain using a custom filter.

How to remove and exclude Bot traffic from Google Analytics?

Well, there is not any foolproof method to exclude or remove the Bot traffic from your Google Analytics report. Here are a few ways you can use to eliminate Bot traffic.

Through Admin View Settings

It should be one of the most straightforward methods one can use to exclude Bot traffic from your website. Go into the Admin View settings on your Google Analytics account. Find the option for Exclude all hits from known bots and spiders under the settings. Make sure that the choice is checked.The setting will block all the instances of Bot traffic that have been identified so far.  Do note that the method would exclude the Bots that have been listed on the International Spiders and Bots list by Interactive Advertising Bureau. Any Bot that hasn’t been listed may go undetected.

Check for Suspicious Traffic

Have an eye on the sources of suspicious traffic to your website. Some of the likely sources can be outdated or obsolete browsers, repeat visits without any apparent session duration or activity, and similar other options. You can exclude the traffic from this particular IP address using an appropriate filter.

Use Referral Exclusion Method

Click on Admin and locate the Property tab. Now choose Tracking Info under it and click on Referral Exclusion List. Once done, exclude each of the domain using the following steps –

• Click on ADD REFERRAL EXCLUSION
• Copy and paste the domain name. You may also enter it manually. Make sure that you have spelled the domain name correctly.
• Click on
• Repeat the procedure for all the domain names you would want to add.

This is one of the most common methods you can use to exclude Bot traffic on your Google Analytics. It should work at the level best to contain the Bot traffic. But, still, it would control the Bot traffic in future, and the historical data will, nevertheless, hold the Bot traffic trend.Please note that the steps involved here would be a little technically advanced. Do not attempt using it if you are not an expert.

The Concluding Thoughts

Bot Traffic has been on the rise in recent times. As we already mentioned, the Bot traffic assumes about 50 percent of the total traffic on your site, and thus it would be essential to contain them if you want your analytics report to be cleaner and error-free.There isn’t a specific and practical solution in place to handle Bot traffic as such. Moreover, taking action against the Bot traffic can only remove them from your Google Analytics report, but cannot guarantee it from hitting your servers and consuming your resources. Effective control can just be possible with the help from the experts. The advanced filtering options would indeed be the best option you can go with. If you are not someone technically knowledgeable, we would advise you to seek advice from the experts.

SEO is an essential aspect of working on your blogs and websites. Of course, most of us take care of the requirements of the SEO related tasks, but there are a few aspects that appear to be unimportant and left out. One such error you may come across can be the Website crawl errors that go unnoticed. One such ‘error’ you may face concerning your WordPress website is a soft 404. How to Fix Soft 404 Error in Google Webmaster Console? Let us take a look at this post.

What Are Soft 404 Errors?

The Soft 404 errors come under the category of the routine 404 Not Found errors. The only difference between the two would be that the 404 Not Found error is returned when the page does not exist at all. But, a soft 404 error does not answer the HTTP 404 status code but shows the ‘page not found’ error.The soft 404 error may also come up when a page that does not exist, redirects the user to a page that is not relevant to the search context. The request is redirected to any page like the Homepage, for instance. In other words, the contents of the web page returned are entirely irrelevant to the HTTP status response that the server responds. Meanwhile, it should also be noted that the Soft 404 error may appear if the page has a thin content.What are the effects of the Soft 404 Error?Well, there are not right issues as such. In the same way, as we discussed the 404 Page Not Found error, the soft 404 error may not have any direct consequences on the ranking of your website as such. However, it can be an issue in the long run.A page may look like a ‘404 Page Not found‘ issue, but it does not need to in the exact sense of the word. Google thinks it can be a concern from the users’ point of view. It is evident that the increased number of Soft 404 errors can be annoying, and disappointing from the visitors’ point of view and thus may need a quick fix.

Fix Soft 404 Error in Google Webmaster Console

How can you detect the existence of Soft 404 error? Well, that should be quite easy to find out. Google alerts you about the presence of soft 404 errors. You would stand to know it either through the search console or email directly.How would you fix it then? The procedure would involve a few steps you need to go through. Let us find the solution step by step –

Step 1 –

• First, Login to your Google Search Console.
• Click on Crawl Errors and then search for “Soft 404 “ on the Crawl Errors tab. They should have separate tags for the errors.
• You will be shown the list of Soft 404 errors if any exist.

Step 2 –

• Click on any of the errors.
• Find what the source of the Soft 404 error is.

Step 3 –

Take corrective action based on the nature and source of the Soft 404 error.It should be noted that there is no specific solution to the error as such. The fixes would depend upon the exact sources for the failure to appear. Some of the possible causes and solutions thereof can be summed up as explained here below.

Possible Fixes depending upon the Causes

The Soft 404 error can be caused due to a host of reasons. We will check out a few causes and arrive at the fixes that can be helpful.

If the Source for error is through WordPress Search

If the Soft 404 error originates from the Search functionality of your WordPress site, it can be due to a spammer trying to spam your site. The best fix to sort this out would be to disable the WordPress Search URL from being crawled.You can do this by editing the robots.txt file. It lists the data that Google can crawl and controls the access available to the Google Crawls. The file is at the root of your site. Download the robots file through FTP, edit it and then re-upload so that the issue would be fixed.You would need to add the following lines to your robots.txt file

Disallow: /?s=

Disallow: /search/

If the page does not exist

You can follow any of the following fixes –

• Create a 404 or 410 error and suggest alternative pages that could be relevant to the search item.
• If you think the page is relevant, create a 301 redirect to a page that has relevant information.
• You may use the .htaccessfile and edit it or opt for a plugin that does the work right away.

If the Page Exist

If you have the relevant page already existing and carries value from the SEO point of view, it would be advisable to check if the page carries enough content. Google Crawls may show up the Soft 404 errors when the page has very little substance. This is referred to as ‘thin content‘ as per the Google standards.If you find the issue in your case is thin content, you can consider adding valuable content to the page. To ensure that the content added is worth the value you would want it to carry.

What should you do after sorting out the errors?

Well, once you have fixed all the Soft 404 errors, the next step you would need to follow would be to let Google know that you have solved the issue concerned. It involves a simple action.Go back to your Google Search Console once again and choose the errors indicated one by one. Mark them as solved. Next, check for the errors once again as you did it in the first step once again. That would offer you the details of any Soft 404 errors left. You will need to perform the same steps once again if you find the similar errors still.

To Recap –

Well, those were some scenarios that can cause the Soft 404 error on your WordPress site. Studying the exact causes and solving them will need you to go through a complete analysis and expert advice. We have attempted providing the outline of the possible causes and fixes so that you can take care of the Soft 404 errors you may be facing.Have you used any other means towards fixing the Soft 404 errors on your site? If you have any alternative solutions, do share your thoughts and experiences with us. Your inputs can help our readers get an added information apart from the one we have provided.If you are a blogger or a website owner, most probably you may be using WordPress as the sole content management tool. Creating your content with WordPress is one of the most straightforward tasks you would be able to perform with WordPress. The sheer number and variety of tools and plugins within the CMS should make it one of the best options for your content management needs. However, some changes you make within site may bring up unwanted results. One such error you may come across is 404 Not Found error message. How would you fix the 404 error message on your WordPress installation? Let us check it out in the following paragraphs.

What causes 404 Not Found Error?

The 404 Error message is displayed when a user clicks on a link, and the page does not exist. In fact, the 404 error message is the default message for a page that does not exist.It can come up as a result of a moved page or removed content. The website you create consists of several pages, posts, tags, categories and other aspects of the content on your WordPress page. You may decide to delete or move some content. There are cases wherein you might have selected to change the URL of a page. Eliminating a tag or category is yet another reason for the appearance of 404 Not Found errors.When a user visits such a page, this is precisely what happens. The user would get a 404 Page Not Found Error. Several external and internal links point to the deleted or modified URL. All these scenarios can result in a 404 Not Found error.

What are the Consequences of 404 Not Found Error?

As a website owner, you may wonder whether the 404 Not Found error is terrible for your website. Well, it isn’t bad for the site as such, but the users who get those messages may get annoyed and leave your site with the inferior user experience.Will it affect your search engine performance? Several views are indicating varied opinions. There are no direct consequences on search engines. However, having too many pages with error messages can be quite disturbing. That would necessitate the need to fix the 404 Not Found error.

How to fix 404 error on your WordPress sites?

Addressing the 404 Not Found error can be quite easy to fix if you can locate the error. Determining the error may not be a tough task as such. You should have access to the Google Webmaster Tool, and it should be the best way to check the pages that have  404 Not Found error.Here is how you can find the pages with 404 Not Found error.

• Log in to your Google Webmaster account.
• Click on the domain name that you want to analyze the 404 Not Found error for.
• Click on Crawl -> Crawl Errors.
• Click on Not Found. You should get the details of all the pages that return 404 Not Found error.
• Click on any of the URLs to find the information on the page with errors.

That way you would be able to find the number of pages that would return you the 404 Not Found error.

Make 301 Redirection

This is the best method to address the 404 Not Found error. The 301 Redirect is a permanent redirection from one of the URLs to another. This can be completely helpful if you have deleted or removed a page or category.You would need to access your .htaccess file for using a 301 redirection. Just add the following code to the .htaccess file, and you are good to go.

redirect 301 /old.html new.html

The syntax should be quite simple to understand. Redirect 301 is the command that asks the search engine that a particular page has been moved permanently. Old.html would refer to the old URL of the page and new.html would apply to the new URL to which you would want the page to redirect to.Please note that a single code may not work with all the pages you have moved. If you have moved or deleted multiple pages, you would need to add a line for each of these changes. Having individual lines for each of the redirects can be the sure shot way to ensure that they are quickly picked up by the search engines.In case you are sure that none of the pages and categories have been moved or deleted, the 404 Not Found errors would have been caused because of several other reasons including the incompatibility with themes or plugins. You can follow any of the following methods so that the error message can be appropriately addressed.

Modifying the .htaccess

You need to edit the .htaccess file and add a snippet code at the beginning of the code. There are several ways it can be achieved. One of the simplest ways to do it would be to use FTP client. Another option you can do it would be to download the .htaccess file and edit it on your word processor. Once done, you can easily upload it back through FTP server or edit it manually.Please beware that changing .htaccess file should be done carefully. It could render your site useless if something goes wrong and fixing it may not be an easy task as such. Ensure that you have backed up the existing .htaccess file before you proceed to make any changes to it.Whichever option you may use, add the following code snippet at the beginning of the .htaccess file.

# BEGIN WordPress

 

RewriteEngine On

RewriteBase /

RewriteRule ^index.php$ - [L]

RewriteCond %{REQUEST_FILENAME} !-f

RewriteCond %{REQUEST_FILENAME} !-d

RewriteRule . /index.php [L]

 

# End WordPress

Reset Permalinks

A permalink refers to the permanent URL link of the post, category and other files. It should be unique in its own right. A pretty permalink refers to the more specific URL. To do this, you can follow the steps here below –

• Access your Admin dashboard.
• Go to Settings ->Permalinks.
• Note the current setting.
• Select the radio button that reads Plain
• Click on Save Changes.

Now, once again click on original pretty permalink and click on Save Changes. That should now reset the permalinks settings you had on your WordPress installation.

Change the Theme to Default

Even after performing the above steps, if you find that the error persists – it could be because of the theme plugin that could be causing the issues. You may go through the following steps to check it out –

• Activate the default theme for your WordPress installation.
• Go into Appearance -> Themes
• Hover your cursor on the default theme and then click on

This will activate the default theme for your site. Check if the 404 Not Found error is resolved. If the issue is resolved, you can safely assume that the theme you have chosen is the culprit. You may consider sharing the information with the developer of the theme.

Check The Plugins

AS we stated before, the Plugins you have installed may also cause some malfunctions and may result in 404 Not Found error. You may consider checking out each of the plugins installed on your system.Follow the steps here below –

• Under your WordPress settings, go to Plugins -> Installed Plugins.
• Click on Deactivate for each of the plugins. You may also consider deactivating all the plugins in bulk.
• Check if the issue has been resolved. If it has, you can conclude that the one or more plugins could have been the root-cause of the problem.
• Now, activate the plugins one by one. Check the functionality after installing each of the plugins.

You can check each of the plugins and find if the issue reappears. If it does, the last plugin you installed is the culprit. You may decide to deactivate the plugin entirely until the issue is resolved. Alternatively, you can communicate the incompatibility with the plugin developer so that they can fix the problem in an update to the plugin.

The Concluding Thoughts

The first step in addressing the 404 Not Found error is to check if you have removed the page for any reason. In case you have done, you can make use of a 301 redirection method so that the issue will be addressed. If you are unable to find the exact reason for the appearance of the 404 Not Found error, and you are sure that you have not deleted any of the pages or categories – you may give a thought to the fixes we have featured here.Please note that most of the fixes here would need you to have at least a basic technical knowledge. Employ all these methods and reboot your system. The issue should be fixed. If you are using any other way to correct the 404 Not Found error, you may share your views and options with us. Your feedback would prove to be helpful for our readers in dealing with the 404 Not Found error securely and straightforwardly.

If you own a business and want to promote your products through a website is the best way to attract a global audience. In case you are handling a huge site, you would indeed use a couple of subdomains, for whatever reasons. The subdomains will be populated on your web host in separate folders. However, you may not want Google to crawl the subdomains. How would you block subdomains from being crawled? Let us find the intricacies involved in subdomains and their blockage from being crawled using “Block Subdomains With Robots.txt” method.

What is Subdomain?

A website does not necessarily mean only the visible part of it – the part that your visitors happen to see. In fact, it consists of much more that. Some of the constituents are complete metadata, XML sitemaps, and other architecture that form part of your site and its framework.Of all the other details, the essential aspects of a website would be the host of complex web pages, and yes – subdomains. The subdomains would assume a vital role in the success of your site regarding SEO.A Subdomain, that part of your website that stands apart from the rest of your site regarding sitemap, but it does not come with a unique URL. A Subdomain is defined as the third level domain that would form part of the top-level domain in a tree structure. For instance, if your site is www.example.com, something like contents.example.com would form part of the subdomain.Most of the website developers do not like adding subdomains to the main sitemap of the website. Such a step would turn the sitemap turn into a complicated mass of data. Using a subdomain would help you set up different sections of your site with their dedicated hierarchy and functionality. This can be helpful for the visitors in getting all the related information. You are free from setting up a new site, and the visitors are spared from the confusion associated with the separate top level domain.A clear differentiation between the main domains and subdomains would be support.yoursite.com and example.com/support. The difference between the subdomain and subfolder should be evident from that example. The subfolder branches off from the top level domain in the site hierarchy.

Why would you need Subdomains?

Subdomains tend to be unique in their own right. You have several distinct apps to cover in addition to multiple products to handle. In such cases, subdomains assume a lot of importance.Subdomains also tend to be equally substantial if you have operations spread across multiple international markets. It would probably explain why the subdomains used by the support of most of the manufacturer websites. Examples can be in.support.exampl.biz and us.support.example.biz. Some companies develop subdomains for different user classes. Customer.example. Command business.example.com are some examples.

Importance of subdomains from SEO point of view

Well, traffic is likely to dwindle if you have your website with a weak structure. Lack of subdomains in above cases where we explained the importance of subdomains could result in a bad layout of your website. Usage of subdomains streamlines the navigation and user experience of your site.But, there is a catch here. Google’s algorithms consider your site and subdomains to be separate from each other. The subdomains will be counted as different sites and ranked differently. That way, having a subdirectory would be beneficial than having a subdomain. Google’s crawling has had issues concerning some specific kind of subdomains.Subdomains can also be helpful in improving the performance of the site regarding SEO if used practically. Some of the advantages of using subdomains can be

• They can help in increasing the insertion of hard to rank keywords. The keywords that cannot be added to the main URL can be added to the subdomains.
• If your site is considerable large and includes a host of different services and products, it can be a great idea to employ subdomains.
• The subdomains can also help you build smaller authority sites effectively.

In case of test use-cases, the best solution would be to block subdomain from being crawled. This is the essence of this article. Let us find how we would able to prevent Google from crawling subdomains.

Block Subdomains With Robots.txt

Well, the most simple and easy to use option for blocking the subdomains from being crawled by Google would be through the robots.txt file. The file is located in the root directory of your top level domain.You need to create a robots.txt file for each of your subdomains. In fact, Google crawlers would look for the robots.txt file in each of the subdomains. Thus, if you want to block the domain to be blocked, you need to insert an appropriate robots.txt file in the root directory of each of the subdomains.Here is how you can do it.

• Open ‘Notepad’ from your programs.
• Create a note with the file extension .txt
• In the notepad document,

User-agent: *Disallow: /

• Save the file

Upload the file to the root directory of each of the domains and subdomains. Ensure that you should exclude the domains that you want to be crawled.You may also create the robots.txt file with the following details –User-agent: GooglebotDisallow: /There can be several other methods involved in blocking the subdomains from being crawled or indexed. However, the robots.txt way should be the best and useful option for your requirements.

The Concluding Thoughts

Of course, the primary purpose of this post was to enlighten our readers on how to block the subdomains from being crawled and indexed by Google, we have also added information on the subdomains and their practical implications. We assume this article will serve towards building a definite knowledge of the concept of subdomains.Subdomains are indeed an utterly positive aspect of working towards the efficient SEO building if employed judiciously. They would help you streamline the navigation functionality of your site considerably. Use them efficiently, but ensure that they are not crawled and indexed by Google algorithms so that your site performance is not adversely affected.