It’s been a while since anyone talked about the size of Google’s index compared to its competitors. My guess would be that it dwarfs Yahoo! when it comes to meat and potatoes pages - non-duplicates, non-100%-template, non-ad-spam - but I don’t know where Google is at for certain. This is definitely one of the things that made Google better than its competitors earlier on though. This could probably spark a big long debate, and I really don’t think there is very much difference between Yahoo! and Google, so I don’t care to go into it. But Google continues to take the lead in finding what’s out there, now starting to spider web forms. Here is a whole new set of candidate web pages that Google will be scoring for search engine rankings that other search engines don’t even bother with. Pure Visibility just helped one client surface some of their pages, hidden behind a form, with Google sitemaps; in the future, these kinds of efforts will probably be unnecessary.

Google announces that it is giving a small number of “high quality” websites this treatment. If that seems elitist to you, well, welcome to the Google algorithm. The number of pages Google decides to crawl and index for a website already depends on the site’s PageRank, so if that doesn’t bother you, this probably shouldn’t bother you either. The experiment is also limited to forms that use a GET method, probably because those pages can be uniquely identified by their URL. It takes a little stretch of the imagination to see them doing it for POST methods. If they wanted to show a page like that in the search results, they’d need to post to it on the fly or else copy the content after spidering and give it a different URL (or come up with something else super creative).

This effort shows Google’s continued dedication to the search engine spider to find the content that’s out there, rather than relying solely on a feed or any other information supplied by a webmaster. Hopefully this branches out to all sorts of websites rather than a chosen few. If you’re not blessed enough to be one of the chosen few, you can still surface web pages hidden behind a GET form using Google sitemaps. You’ll want to make sure you have great on the page optimization for anything found solely through Google Sitemaps if you plan on seeing it on a search results page.

One important question that arises is how Google will score a page behind a form for search engine rankings. Link popularity is super-important for good rankings in Google, and a page behind a form doesn’t have any static links pointing to it. My guess is these pages would only show for rare queries that don’t have a lot of possible candidates, but who knows . . . maybe eBay will start showing for everything because they have search forms everywhere in their site.