You are here
Home > Google

Is Googlebot unnecessarily crawling a large number of distinct URLs on your site?

Have you seen this message? “Googlebot encountered an extremely high number of links on your site.” I have and more than once and more than twice. The main problem with organic internet marketing is understanding that the core, the foundation, of your success is technology and having Google to go through unnecessarily high numbers of URLs can be caused by a variety of problems for your online marketing efforts. One of them being that Google just gives up and throws your web out.

If your URLS look like this:

http://www.example.com/search/noheaders?click=6EE2BF1AF6A3D705D5561B7C3564D9C2&clickPage=OPD+Product+Page&cat=79

Start by getting a Google Webmaster account and activate it, implement it on your site. Then look at your data as it is collected, you might actually get an email from Google Support saying something like:

“Googlebot encountered an extremely high number of links on your site. This may indicate a problem with your site’s URL structure. Googlebot may unnecessarily be crawling a large number of distinct URLs that point to identical or similar content, or crawling parts of your site that are not intended to be crawled by Googlebot. As a result Googlebot may consume much more bandwidth than necessary, or may be unable to completely index all of the content on your site.”

If this is the case ACT on it, this is Google giving you heads up before the real problem starts.

Remember that the best solution is to choose a good search engine friendly web CMS when you start your online presence or you are reinventing your company online.

Be Sociable, Share!
Optimize Your Web
Kristjan Mar Hauksson is the founder and director of search and online communications at Nordic eMarketing. The company specializes in multilingual online communications, organic search engine optimization, and marketing through several verticals such as tourism, finance, government, and pharmaceuticals. It helps companies gain international visibility online and to use the Internet as a communication channel; it also provides consultation in web content management systems and analytics solutions. Hauksson is on the board of directors of SEMPO, on the Advisory board for Bing Advertising and founded the Iceland SEO/SEM forum. He has been involved in developing Internet solutions since 1996, and involved in search engine optimization and marketing since 1997. He is a publish author on the topic of Internet Marketing and among other co-authored the acclaimed Global Search Engine Marketing.
http://www.nordicemarketing.com

3 thoughts on “Is Googlebot unnecessarily crawling a large number of distinct URLs on your site?

  1. Glad to see your post, and I have received the warning recently. Does it mean I cannot use the complicated URL format any more? As you know the web pages about search have such kind of URLs normally. Or we just tell the robot not to crawl these part of content. Any suggestions plz?

Comments are closed.

Top