SEO Information

Abandoned by Google! Googlebot, Wherefore Art Thou Googlebot?


As a search engine optimization specialist I often optimizeexisting web pages for small business clients, upload them tothe site and see pages re-indexed by Google within a week.This only happens with existing business sites that have beenonline for a few years. Google seems to be updating theirindex as often as every other week at this point and olderestablished sites that are already indexed seem to be re-crawled on that twice a month schedule on a fairly routinebasis.

Two clients that hired me for recent work saw their rankingsshoot to the top for a newly targeted search phrase in aweekend when I did optimization on a Thursday and they wereranked instantly by Saturday. Now keep in mind that thisdoesn't happen for everyone, only those that have been onlinefor some period and already have significant content thatsimply needs tweaking and proper title and metatag informationadded. They usually have relatively good existing PageRank anddo well for other RELEVANT search phrases already. I offer thatwarning only to avoid instilling false hopes in anyone hopingto achieve the same instant ranking boost overnight.

Those clients that do succeed in this way are often thrilledwith the results accomplished in such short order. I'd loveto be able to offer that type of ranking boosts to everyone,but some are more equal than others when it comes to easy,inexpensive SEO tune-ups that rev up your rankings overnight.Your mileage may vary.

WHY DO NEW SITES SUFFER?

What is going on with newer sites that don't get crawled formonths? I've got a client, a newer attorney directory thatoffers tons of great information in the form of articles onspecific areas of law, links to incredibly valuable andrelevant legal sites and over 600,000 attorneys listed bypractice area and state. Yet the site has not been re-crawledby Google for over 3 months! Now this would not be such a bigissue for many sites, but this site is relatively new and we'veoptimized all the titles, tags & page text, created a completesite map and placed links to all these resources on the frontpage.

I know that the site is not being crawled because Google'scached copy of the front page shows it before we did thework four months ago, without the new links and withouttitle tags. We've submitted the site by hand, (manually)once a month for three months via the Google Add URL page.http://www.google.com/addurl.html When the hand submissionfailed to get it re-indexed for four months, we submittedthe sitemap page, which has not been crawled at all. Googleshows only ONE page on this site, when in fact it hasthousands of pages, a sitemap and dozens static pages!

Part of the problem is that this site must be dynamic, sincea database of over 632,000 attorneys must be accessed,retrieved and served for any of those law firms searched forto be returned to the site visitor. Google warns owners ofdynamic sites that Googlebot may not crawl dynamicallygenerated pages with "?"" question marks in the URL. This isto avoid crashing the server with too many concurrent pagerequests from Google's spider.http://www.google.com/webmasters/2.html#A1

The solution to this dynamic URL problem has been discussedwidely in search engine forums and solutions have been bandiedabout including software provided by SEO's, URL re-writetechniques for dynamic pages on APACHE servershttp://www.alistapart.com/articles/urls/ and PHP pageshttp://www.stargeek.com/php-seo.php to generate search enginefriendly URL's. Others recommend simply adding static HTMLsitemap pages as alternatives for the search engine spiders.In this instance the client's developer simply said "Ican'tdo that (PHP solution) on this server". So we resorted toputting up the static HTML sitemap pages with hard-coded

URLS to the main 54 pages of the site athttp://lawfirm411.com/Law-Firm-411-sitemap.html This shouldget at least those fifty pages crawled by Googlebot, butGoogles' spider appears not to be crawling this site at all.How do we know this? See for yourself by using the followingquery in the search box at Google: allinurl:www.lawfirm411.comwhere the result page shows ONE page in the results. If youtry that query on your own site (replace your own domain namefor lawfirm411.com), you'll see the results lists ALL yourpages.

The site home page was crawled by Google four months ago, whenthey took their "Cached Snapshot" of the page. You can seethis by visiting the Google cached page here:http://66.102.7.104/search?sourceid=navclient&;ie=UTF-8&q=cache:www.lawfirm411.comwhere the date of this snapshot is "Apr 20, 2004 07:42:19 GMT"and they haven't been back since. The page in that snapshothas none of the newly added links, an outdated title tag, andold content.

This problem is not unique to this site. One client we workedwith two years ago had a dynamically generated, framed site!Those two site structures have always given search enginestrouble. Their site was not crawled at all and only the frontpage showed up. Our solution was to create a second domain(owned by the client), which had static HTML pages thatprecisely mirrored the content of the client's framed,dynamically generated site. Guess what happened afterGooglebot crawled the static site? Google indexed the framedsite in full and then banned the static site from the index!Not an approach we advocate, but the one that worked for thisclient.

We're still searching for ways to get Googlebot back toLawFirm411.com before creating that new static site, butdecided to share this odd experience with the SEO communitybefore going to any extremes. Google provides over 70% ofmost search engine referred traffic to ALL of our clientsand we realized we can't site idly by and see a major clientlanguish because Googlebot didn't like what it found at theclient site on the first visit four months ago.This issue dogs newer sites in other places as well. The OpenDirectory Project has also become notoriously slow in addingnew sites to the directory and in this case, has not pickedup this site even after 6 regular monthly submissions. Theweb playing field may have begun tilting toward older,established sites and away from new ones.

------------------------------------------------------------

Mike Banks Valentine is the SEO for http://www.lawfirm411.com

Contact him at http://www.seoptimism.com/SEO_Contact.htm

Improve Your Small Business Online at our Ecommerce Tutorial

http://website101.com/Free-Tutorials/index.html

------------------------------------------------------------


MORE RESOURCES:

PR Web

SEO, Why You Are Doing it Wrong
Search Engine Watch
What follows is a cautionary tale of missteps towards the complex problem of ranking on search engines and an examination into why small businesses are completely at a loss as to what constitutes “ethical” SEO. Before he was worried about Google and ...
Google Releases New Penguin Update, Numerous Online Sites AffectedSan Francisco Chronicle (press release)
Microsoft And Others Serve Up Google Panda And Penguin AdviceMediaPost Communications
Google Penguin - A Lesson It Teaches [Infographic]SubmitinME
Travolution -Hit Search -Search Engine Land
all 17 news articles »


SEOReviews.co Offers Consumer Reviews of Leading SEO Services
Seattle Post Intelligencer
In today's marketing environment, many businesses find it essential to invest in an SEO service, but finding the right SEO service can often prove challenging. A new site, called SEOReviews.co, is seeking to provide objective information and customer ...

and more »


SEO Miracle Presents Platinum SEO Service Package
Albany Times Union
SEO Miracle has decided to present its new and final SEO service package for this year called the Platinum package. An SEO service has taken a totally different meaning since Google presented its PageRank algorithm. Ever since then, website owners are ...

and more »


SEO Positive Announce Acquisition with Leading Snail Racing Company
PR Web (press release)
The search team at SEO Positive are pleased to announce the acquisition of a new contract with the UK's top snail racing firm Fundeo Limited. Based in Edinburgh, Fundeo specialise in providing their customers with a new and entertaining way to enjoy ...

and more »


PR Leap (press release)

SEO Advantage Barack Obama Over Mitt Romney
PR Leap (press release)
The amount of traffic to Barack Obama's website as opposed to Mitt Romney's can be attributed to the SEO of the candidate's websites. Mitt Romney's website has only 16% of the amount of external followed link Barack Obama's website has.

and more »


SEO & SMO Technology Masterclass 'On Demand' at One Wimple Street
ITCM
One Wimpole Street's latest free technology masterclass on Thursday 17th May gave event organisers the opportunity to find out how social media optimisation benefits business and why search engine optimisation (SEO) is so important when it comes to ...



ExtremeSocialMedia.org Combines SEO and Social Media to Bring Businesses ...
SBWire (press release)
One company that has been helping other businesses achieve consistent page one rankings is an Austin SEO company called Extreme Social Media. Extreme Social Media have seen firsthand what page one listings can do for businesses.
COSTA RICA NEW MOBILE APP BECOMES YOUR PERSONAL TRAVEL AGENTTravPR.com (press release)

all 3 news articles »


Entrepreneur

SEO vs. PPC: Knowing Which Is Right for Your Website
Entrepreneur
BY AJ Kumar | May 21, 2012| When it comes to boosting traffic to your website, you have two basic options: pay-per-click (PPC) advertising or search engine optimization (SEO). You can pay for traffic using the PPC advertising programs provided by ...
Learn: SEO Lingo DefinedSearch Engine Journal
29 Prime Enhances Client Websites with New GPS Location FeatureSan Francisco Chronicle (press release)
SEO Miracle Allows Users To Check Google Rankings In CanadaRedOrbit
Albany Times Union -iTWire (press release) -Search Engine Land
all 24 news articles »


Fusion SEO Launches SMART Capture SEO™, Striving to Expand Upon the Maryland ...
Virtual-Strategy Magazine
Evan Featherstone, President of Maryland-based Fusion SEO, has launched SMART Capture SEO™, with the vision of having Fusion SEO be the future benchmark of local, results-based web marketing consulting. Baltimore, MD, May 22, 2012 --(PR.com)-- Evan ...

and more »


SEO Project Manager
Bizcommunity.com
We are seeking the skills of a self-starting SEO project manager to manage and build an SEO team with an aim to rank our websites for multiple highly competitive casino related keyword terms and phrases. You must have strong people and project ...


Google News

home | site map
© 2006