Oodle Blog

GoogleBase

Today, a few articles have surfaced that Google will enter the classifieds business.  Very cool looking product.   We’ll definitely include Google’s listings in our index (unless they stop us which I very much doubt) .  The more sources the better.

Based on previous reports, it now appears that Google is pursuing a dual strategy with respect to classifieds.  They will both a)  become a classifieds publisher by directly taking listings and b) incorporate classified listings published by other sources into Google Search, Google Local and Froogle.    This dual position – as both a publisher and a search engine – is likely to spark a big debate. 

Search should be a win/win game.  Publishers get free traffic (their advertisers in turn get more prospects) and consumers get a better user experience.    When a search engine sells their own listings, they begin to hurt their partners by competing against their core business.  For this reason, Oodle doesn’t take listings.  It will be interesting to see this play out.  If nothing else, Google is certainly exhibiting portal-like behavior.  Stay tuned, I'll be tracking the debate...

Moving Forward

We’re very excited about where things are going.   Oodle provides a valuable service to consumers by enabling them to see all their choices.   We’ve seen over 25% growth in users and traffic to partners every month since we launched.   Last month alone we sent Craigslist well over 1 million referrals (i.e., users sent to their site).   We also continue to have the largest classifieds index (Craigslist represented about 20%) and will continue to add millions of more listings to our index in the coming months.   This week, we received over a dozen requests from classified sites wishing to be included.

This whole Craigslist thing has unfortunately mushroomed into quite a big deal.  They certainly have every right not to be in our index and I’m genuinely sorry that we appear to have upset them.  We continue to be great fans.

In a recent news article, Craigslist CEO Jim Buckmaster made a couple of comments about our actions.  It’s never good to see these sorts of things play out in a public debate but we wanted to provide our point of view.

We have always been up front with Craigslist.  Before we launched, we sent them a note asking them to check out our site.   A month later we requested a meeting to discuss what we were doing and how we were accessing their site.   Both times they politely declined to meet with us.

We do our best to act as a responsible search engine.  We strictly follow the robots.txt protocol and check it every time we visit a site.  For those sites that don’t want to be included, they simply need to add a single line of text to their robots.txt file (as of today, we are still not in Craigslist’s robot.txt file).

We did check Craigslist’s terms of use agreement when we started crawling their site and it allowed such activities.  It appears that it recently changed and I’m sorry that we missed this. There would be no billion-page indexes if search engines needed to regularly review the terms of use agreement of every site in its index (that’s why the robots.txt protocol exists).

After Jim let us know that Craigslist wished to be removed from Oodle, we complied within two business days.  During this time, we tried to establish a dialog to address their concerns and are still interested in doing so.

We always try to be very mindful in the way we index a site.  We go to great lengths to minimize our impact on our partners’ servers (and visit most sites in the middle of the night when the cost to crawl is near zero).   To put it in perspective, Google and Yahoo pull about three times the data from Oodle's servers each day than we pulled from Craiglist.  It’s helpful to know the issue at hand is the load we place on their servers.  We have a number of proposals on how to address their concerns.  I hope we get a chance to share them.

Where's Craiglist? part II

I wanted to briefly respond to a few of the comments I've read on this issue...   

I think it’s important to keep in mind what's best for consumers.  And I think being open is good for consumers.

People tell us they want to see all their options.  This is especially important for classifieds: 

• the market is fragmented – lots of listings, in lots of places, always changing
• listings are perishable inventories of one – the items are generally unique and will be gone quickly (a job opening, available apartment, used car)

If I'm looking for an apartment and don't see the listing for my dream place, it's gone and I never even knew about it.  If I am looking for a used Volvo, I want to see all the listings available as soon as possible to maximize the potential for finding what I want.   If I collect rare coins, I want to be alerted on listings across as many places to buy as possible.

Comparatively, this is much less of an issue when I'm shopping for a new digital camera (where I can go into any one of ten stores and buy what I want). In this situation, search still has value to empower users in price comparison -- causing retailers to further differentiate in service, convenience and support (all benefits to users as well).

A similar argument can be made for consumers trying to sell items.  They want to reach the biggest audience so they can get the best deal.  Services like Oodle, that help bring prospects to a consumer's listing, are helping them not hurting them.

Craigslist certainly has a right to not be included in Oodle.  That said, we believe that by helping consumers find listings on Craigslist, we positively contribute to their service.

Where's Craigslist?

We are no longer adding new Craigslist listings to our index.   We received a request from them to do so and we honored that request.   We’ll miss Craigslist.   They are clearly a good service and we're big fans.  We are working to understand and address their concerns.

As of last month, about 80% of our listings came from sources other than Craigslist.   Without them, we still have about 4.5M active classified listings and that number will continue to climb by the millions in the coming months.

We strongly believe that search plays an important role in any Internet ecosystem, especially ones involving commerce.   We’re glad to have had a good response from the 99.9% of the other classified publishers in our index and we’ll continue to work hard to make them happy.  We send them free traffic.   We don’t compete with them by taking listings.   And we'll continue to work hard to make classifieds a better online medium for consumers trying to buy & rent things.

At the end of the day -- with our without Oodle -- consumers check multiple sources when using classifieds.  Oodle  makes this process less painful and helps consumers to be more successful.    And if you're in the business of publishing classifieds, this is a good thing.

Web 2.0 speak

In preparation for this week's show, I thought it would be fun to describe Oodle using Web2.0 buzzwords…
 
Oodle indexes classifieds listings from across the web, many of them without even a URL to call their own. Once we find them, we tag them (e.g., pet listings with breed, gender and color); remix them with other data sources (e.g, expand abbreviations, normalize location); and create mashups with other web services (e.g, put them on Google Maps using AJAX). 

 In doing this, we enable a long tail for classifieds. Prosumers can easily find listings not only from big providers, but from small community sources like local newspapers and niche national sites such as mykidscloset or trucker-to-trucker.

 Our business model, while still unfolding, will continue to be defined by lightweight, decentralized business relationships – Ad Sense, content syndication (XML/RSS) and affiliate partnerships.

Perhaps by next year we’ll even incorporate an architecture of participation ;)