Why Google does not index your website

Why Google does not index your pages

Solving problems with Google indexation and missing pages

Why can Google not find my website? Nothing is is as annoying as working on a website and finding out it does not get indexed. Even though being found in Google can massively improve your online success.

Aside from your website not being indexed, there is another possibility. That is that your website is actually indexed, but ranks very low in the search results. In this case you need to look are search engine optimisation (SEO). To rank higher in the search results there are many possibilites, like content optimisation and linkbuilding.

Website indexation control

To check if your website is in the search results, you can do an indexation check. This checks if your website can be found in the search results. When it shows that your site is not being indexed, the best option is to use Google Search Console. That gives you the option to request indexation and track problems.

Why is my website not being found?

If after the site indexation check you still ask yourself:”why does Google not index my site?” we would gladly help you. If your website does not appear in the search results, there can be many different causes. Below, 13 different causes that might be the cause of your site not being indexed are listed.

1.    Google has not found your website yet

Especially when you just have a new site, the possibility exists that the website indexation is not done yet. Indexing a website can go quite fast, however this is not always the case. You can help Google by putting a link of your site on a relevant popular site. This will cause Google to find you earlier. While you are doing this do not forget to make content.

2.    Google cannot access all pages

When Google cannot access all parts of your website, it cannot index the pages in those parts. A clear navigational structure ensures that the Google robot, that indexes pages, can find the pages easier. Try categorizing your products. Also, do not hide pages in Flash or Javascript.

3.    Your website is in the Google sandbox

 It is possible that your website was found, but can not be found anymore. There is a big chance it ended up in the ‘Google Sandbox’. Even though Google denies the existence of the Google Sandbox, there is enough evidence showing its existence. The goal of the Google Sandbox is that website first need to prove themselves before they are indexed.

The problem can lasts weeks (even months),but there is no other solution than waiting. Meanwhile you can continue expanding, promoting and optimizing your website.

The Sandbox issue is most common with highly competitive search queries, like brand, business or product names. Long search queries (or long tail keywords) are often hit hard. Search engine optimization and linkbuilding have no influence on the Google Sandbox, though these actions never hurt your site. It should not be confused with a Google penalty (number 4).

4.    Google Penalty

Aside from keyword stuffing there can be many reasons to give your site a Google penalty. Because of this, your site will not be indexed. In your Google webmaster tools (Search Console) you can check if you have received a penalty under the heading: ‘Manual Actions’.

5.Not in the sitemap.xml

Another crawling mistake is that your sitemap is not updated and the old/damaged version keeps being used in Google Webmaster Tools. Always check if you have handled the problems as shown on the dashboard of Webmaster Tools, or if you have executed a new sitemap, then enter the sitemap again.

6.    You forbid Google to index pages

Inside a so-called robots.txt file, you can indicate that you do not want certain pages to show up in the search results. For example, when your robots.txt includes a rule like this one;

User-agent: *
Disallow://

Check your robots.txt files on unintended exclusions.

7.No-index tag

Aside from excluding pages on a general level (number 6), you can also exclude pages on page level. You have to avoid the robots noindex tag. The metatag: <meta name=”robots” content=”noindex”> prevents that most search engine web crawlers index this page of your website. When this unwanted, you have to remove the metatag from the <head> of the page.

8.Duplicate content (double content)

 

Duplicate website content is not appreciated by Google. Try to avoid large chunks of text that are the same of similar to each other. The parts of duplicate text do not even have to be meant to be misleading. For example, print versions of web pages.

When Google notices that duplicate content is used on purpose, your pages or even your whole website can be removed from the search results. Use canonical URL’s to indicate which page has the preferred URL.

9.Wrong canonical

Make sure that canonical URL’s are indicated in the correct manner. if you do not explicitly show which URL’s are canonical, Google will make the choice or will put the same weight on both URL’s. This can have a negative effect on the indexation of your website. To check which URL is seen as the canonical URL, you can use the URL-inspection tool of Google webmasters.

10.Website does not index because of server

Server settings can be the cause of your indexation problems. Per type of server (Apache, Nginx, IIS, etc.) there can be possibilities to stop pages or bots. Even though it does not happen much that a Googlebot is blocked from the server, it is a possibility.

When you have problems with your server, also check the .htaccess file. This prevents the crawlers of indexing your website.

Besides the bad user experience, a slow website increases chances on the possibility that the site is offline when the Google bot visits. Not all hosting providers have a high up-time and enough capacity on the servers.

11.  Crawling budget

Indexing a website by Google asks capacity from the ‘Google crawl robot’. per website a certain capacity is reserved. When your website has a lot of pages, it can happen that (because of the limited crawling budget) it will take longer. How many pages are crawled per day by Google depends on a couple of external factors, like the number of backlinks, website errors an the authority of your website.

12.  Wrong redirects to your new website

When you recently switched to a secured SSL-connection or a new domain name, it is possible the settings are not correct. All old URL’s need to be redirected with a 301 to the new URL. This is also the case with a new menu structure. When a page cannot be found anymore, you will see a 404-error.

13.  Platform related

When your website does net get indexed, it can also be because of the platform.

Shopify not indexed

A clear reason why you are not found with an online Shopify website is because your online store is secured with a password. Aside from that, a Shopify trial account can also be the reason why your website is not indexed. When you use a payed account the pages are added for indexation.

WordPress site not indexed

You can switch off indexation in WordPress by clicking the checkbox. This is probably not what you wanted to do. To check this go to ‘settings’, here you will find ‘search engine visibility’. The option ‘ block search engines from indexing this site’ needs to be off so your website can be indexed.