We’re very excited about where things are going. Oodle provides a valuable service to consumers by enabling them to see all their choices. We’ve seen over 25% growth in users and traffic to partners every month since we launched. Last month alone we sent Craigslist well over 1 million referrals (i.e., users sent to their site). We also continue to have the largest classifieds index (Craigslist represented about 20%) and will continue to add millions of more listings to our index in the coming months. This week, we received over a dozen requests from classified sites wishing to be included.
This whole Craigslist thing has unfortunately mushroomed into quite a big deal. They certainly have every right not to be in our index and I’m genuinely sorry that we appear to have upset them. We continue to be great fans.
In a recent news article, Craigslist CEO Jim Buckmaster made a couple of comments about our actions. It’s never good to see these sorts of things play out in a public debate but we wanted to provide our point of view.
We have always been up front with Craigslist. Before we launched, we sent them a note asking them to check out our site. A month later we requested a meeting to discuss what we were doing and how we were accessing their site. Both times they politely declined to meet with us.
We do our best to act as a responsible search engine. We strictly follow the robots.txt protocol and check it every time we visit a site. For those sites that don’t want to be included, they simply need to add a single line of text to their robots.txt file (as of today, we are still not in Craigslist’s robot.txt file).
We did check Craigslist’s terms of use agreement when we started crawling their site and it allowed such activities. It appears that it recently changed and I’m sorry that we missed this. There would be no billion-page indexes if search engines needed to regularly review the terms of use agreement of every site in its index (that’s why the robots.txt protocol exists).
After Jim let us know that Craigslist wished to be removed from Oodle, we complied within two business days. During this time, we tried to establish a dialog to address their concerns and are still interested in doing so.
We always try to be very mindful in the way we index a site. We go to great lengths to minimize our impact on our partners’ servers (and visit most sites in the middle of the night when the cost to crawl is near zero). To put it in perspective, Google and Yahoo pull about three times the data from Oodle's servers each day than we pulled from Craiglist. It’s helpful to know the issue at hand is the load we place on their servers. We have a number of proposals on how to address their concerns. I hope we get a chance to share them.
Was interested to read the inman.com article, but the site requires paid registration. Any way that you could post some important quotes or perhaps another source?
Anyhow, not at all surpised to see that this was a Jim-thing... From what I gather, Craig dosn't play as much of a day to day role in site operations/policies as he used to, leaving Jim in charge. IMHO, Craiglist has been on the downward spiral since Jim took over, it appears that he has gone off the track of the orginal mission. It's too bad.
Posted by: Joey Boots | October 19, 2005 at 02:06 AM
It's too bad that Jim has time for press interviews on the matter, but can't be bothered to speak with you directly. Oh well.
Forget about CL - Oodle is now in the perfect position to start accepting it's own classified ads, alongside the ones from other sites. Take this opportunity dude, and slay the dragon. I've been waiting for someone to create a site that is in a real position to compete with CL, go for it, dude. :)
Posted by: FrankM | October 19, 2005 at 02:28 AM
Good luck in trying to work out a solution with Buckmaster in once again including CL in your listings. I supect though that you will not be successful regardless of the accomodations you are willing to make. In my opinion and probably yours (although at this point you are probably hesitant to say so publicaly) CL is just using the "load" argument as a pretext.
Posted by: Gregg H. | October 19, 2005 at 02:44 PM
I would see it as a compliment.
They would only refuse the one who is taking more than giving.
Posted by: robert | October 20, 2005 at 06:14 AM
Any monopoly is "no outlet" way for business.(sample: Google) I think oodle.com have a goal create own business on the classifieds's shoulders. It's so easy to take traffic using others content for free and after create own ads submission page.
Aggregators must die!
Posted by: Alex | October 20, 2005 at 08:04 AM
lol, How do you think CL gets it's content? They rely on people to post free ads = free content for CL. If you really wanna push it, then what about commerce sites that make requests for and publish product reviews by customers? That's free content too. But stuff like that is useful, right?
Every search engine out there is an "aggregator"... don't you use search engines? Makes the web more useful, hmm?
I really don't think what Ooodle is doing is harmful. And evidently most of the classified sites that it links to are glad to be part of it.
Posted by: FrankM | October 20, 2005 at 08:28 AM
"And evidently most of the classified sites that it links to are glad to be part of it."
------------------------------------
I'm not sure absolutely. Only classifieds who doesn't see so far in future or "handmade" sites. We will see what happen.
Posted by: Alex | October 20, 2005 at 08:40 AM
A new atricle about this was just posted on ClickZ.com.
http://www.clickz.com/experts/brand/buzz/article.php/3558091
"Craigslist users and their listings are not a commodity to be traded upon for commercial purposes," Buckmaster told me in an e-mail exchange. "Robotically harvesting content from our site compromises our own users' access to finite server resources, while providing little or no offsetting benefits."
--Here he gives 2 reasons. The first one is the real reason, but for those that don't like that reason, he offers the second phony reason.
Posted by: Gregg H. | October 20, 2005 at 07:51 PM
Thanks for the link... Jim's gibberish was hillarious to read, spoken like a true CEO, lol. Thing is, if CL was just starting out now (and didn't already have a monopoly) they'd be welcoming Oodle with open arms.
Posted by: FrankM | October 20, 2005 at 09:56 PM
Sorry, one more... I found this interesting since CL offers RSS feeds for personal use anyways...
"Robotically harvesting content from our site compromises our own users' access to finite server resources, while providing little or no offsetting benefits."
Posted by: FrankM | October 20, 2005 at 10:13 PM
"As Oodle founder and CEO Craig Donato pointed out to me, eBay now offers an API ... Many travel players, ... pay affiliates ... when people make reservations after being referred from those sites."
----------------------------------------
... and the affiliates sites create a information "noise" in Internet like a forest of road sings completely closed the road. It's make sense only for seller and army of parasites who make money from nothing by creating so far link between buyer and seller.
Posted by: Alex | October 21, 2005 at 09:56 AM
This has nothing to do with your service but Jim is a looser.
Posted by: Lima | October 21, 2005 at 01:06 PM
Ha. Here's to the concept that even bad publicity is good publicity. I didn't even know oodle.com existed until I read Yahoo's article (linked to their front page.)
Now that I've checked out oodle and what you do - I love it! oodle provides a service that saves time and provides a wider spectrum of resources for the user.
Scraping? If this is scraping - then Google, Yahoo and every other search engine are scrapers too.
My guess is -- oodle is becoming a contender and certain 'someones' are feeling threatened. Keep up the good work. I agree with whoever it was up there that said - Go ahead - now is your opportunity. Slay the dragon.
Posted by: Mina | October 21, 2005 at 04:54 PM
oodle > cl
craigslist died years ago ... what exists now is crap.
jeremylanghans.com
Posted by: jeremylanghans.com | October 22, 2005 at 07:04 AM
I thought you said "Moving Forward?" Stop obsessing about CL!
Posted by: Drew | October 25, 2005 at 12:14 PM
I am kind of shocked that you guys seem indignant about Craigslist not wishing you to scrape their site and make money from their hard work. You seem to think that you are providing this great service and it is all about the consumer but you are obviously making money from this concept otherwise you wouldn't be doing it.
Posted by: Jason Bentley | November 22, 2005 at 07:12 AM
Jason,
I'm not indignant. I'm surprised and dissappointed.
I do think we have a good service that is valuable to consumers. I don't think there is a conflict in building a valuable service and trying to make money. I think Google is a great service and they make a lot of money.
Posted by: Craig Donato | November 30, 2005 at 11:55 AM