Last week, I sat down with the founders of ShopWiki; Kevin Ryan, Dwight Merriman, and Eliot Horrowitz. If you recognize the name Kevin Ryan, it’s because he is the former CEO of DoubleClick. Dwight is the former CTO of DoubleClick. Eliot was a software engineer at DoubleClick. Read full bios here.
Talking with the ShopWiki founders was like talking to old friends because from the start, we were speaking the same language. All the issues that I’ve been harping on for the last 6 months – comprehensiveness, price floors, lack of loyalty/poor user experience – were dismissed as we discussed how to create a better shopping experience and provide relevant results to users.
Relevance. That’s what this comes down to. Some of the current engines are not consistently providing relevant results to users. If 35 – 45% of revenue comes from people clicking on Google AdSense ads as opposed to results aggregated through data feeds (your main business), isn’t that an indication that there’s a lot of room for improvement?
Now I’m not saying that ShopWiki is the answer. ShopWiki is far from perfect and the service has a long long way to go, but it’s on the right path and is a long term threat to the established players who are not thinking seriously about crawling.
I’ll post specific comments/criticisms on ShopWiki very soon, but let’s just get the facts out there. Following is my interview with the founders of ShopWiki; Kevin, Dwight, and Eliot.
“Dwight and I spent months going through different verticals. Comparison shopping was the biggest from the beginning and the problem hasn’t yet been solved. With travel, you ask 100 people where to go and 50 would say Expedia. In comparison shopping, ask a hundred people and not more than 5 say I use [comparison shopping site] X. You already have 6 engines out there which are incredibly profitable. For us, we’re creating a better product.”
“A lot of stores carry common items and with 5000 merchants submitting data feeds, consumers will get good results. But virtually everyone has a niche interest or hobby which produces a subset of queries for more specialty items like cycling parts and accessories. On a Shopping.com search, you won’t find much. Most people have areas like that which they are looking for where having a bigger set of information proves to be helpful. Half the time current shopping search results [on the other engines] are ok. Half of the time, the results are lacking.”
What are the problems with the current engines?
-Not terribly consumer friendly [results are often biased]
-Search can be done better [don’t always return great results, you can’t type in 5 megapixel digital camera with 10x optical zoom]
-Existing sites don’t help you with the early stage of shopping – what to buy – that’s where the Wikis come in
Will you take data feeds?
“Data feeds work for the sites that will send you a data feed. The problem is the long tail. If you want to be comprehensive, you have to crawl. The question then becomes should we crawl and take data feeds. If the crawling doesn’t work, we’re not going to be accurate. If the crawling works, why would we need data feeds? There are a lot of administrative costs involved with accepting data feeds [time, resources for sales, formatting, submitting etc.] and you need a pretty large department to take care of it.”
What is a store on ShopWiki defined as?
“A store is defined as a merchant from whom you can buy something directly. If you have to call the store to place an order, we might still list it in a way that makes sense for the user.”
“We’ve done head to head searches, and we have more stores than the other shopping engines. We keep affiliate sites out. We have an affiliate detector. The vast majority of the 120,000 merchants are real. We don’t find stores listed on the other shopping comparison engines which we don’t have.”
What about in stock vs. out of stock listings, tax, shipping?
“We show you what’s on the site. If it says in stock, we’ll say in stock. If it says out of stock, we’ll say out of stock. That is something that will get better. Sometimes a consumer will search for a rare product which is only carried by one store. As opposed to saying that product doesn’t exist and not listing it on ShopWiki, we will list it as out of stock because that would be a better user experience.”
“We claim to have more stores than anyone else. We do not claim that [our] stock status or shipping and tax information is better. Once the site is live, merchants will be able to come in and configure that information.”
What about auctions?
“Right now we’re indexing BuyItNow listings (which are more like conventional offers), but not auctions. We’ll re-visit auctions later. It might be a solvable problem, but you have to do it without causing clutter.”
How often do you crawl?
“It depends on the site. [The frequency] can range from daily to 48 hrs to 3 weeks.
Merchants don’t want us crawling every day, and we can be smart about it. We know when to come back to visit a site. Out of stock listings are a problem for a very small number of items.”
I’ve heard that there are up to 300,000 stores on the web. Could you go much bigger than 120k?
“I can’t believe there are 300,000 stores online in the US. Our system has inspected 350,000 sites and determined if those sites are or are not stores. Many of these sites are spam, affiliates, or promotional sites, but you can’t buy a product there. Other sites are services. We’re not excluding services, but those are not categories where we claim to be great or included in the 120,000 stores.”
What has changed that allows us to accurately extract data from so many merchants? When the first shopping comparison engine were made, the developers were talking about AI, claiming their crawlers could learn to interpret the data on the page, but websites change, there are often multiple prices on the page, each site is different, etc. Are we past all that?
“This is a hard technical problem to solve. Crawling is easy, but we’re taking a very different approach to data extraction. The first [data extractors] had a standard AI data mining approach. What we did is a little crazy. We have an unorthodox approach to the problem. It’s analogous to Google’s work. When Google came up with PageRank, it was a completely different approach to the ranking problem, and the company got better results. It wasn’t that no one could have done it 3 years earlier; it’s just that no one actually did it.”
“It’s not perfect, but when we have a higher accuracy level [for search], and you overlay that with a much bigger index, it’s a net benefit for the user.”
Was the technology built in house?
How are the results determined?
“We want to be like traditional search engines, but we also want to tell people if it’s a good or bad site. [ShopWiki lists ‘Most Popular’ Stores first. There are 50 or so ‘Most Popular’ Stores. These are the Amazons of the world.] If a site returns really slow results, we put a snail next to the listing. We’re not trying to be friends of the merchants. We’re consumer friendly rather than being an extension of the merchant. Most shopping comparison sites today are on the sell side. We want to be on the buy side.”
How will you market?
“We’ll rely on PR. Right now, we’re 100% focused on getting the right product out there. In the fall, we’ll start on the marketing front. We’ll buy ads, keywords, and see how it works. The focus has to be on the product.”
Where do you have to improve?
1 – There are 1000 little features or developments that can make searches better. We have to improve SKU consolidation, for instance. We’ll work on these things over the next several months.
2 – There are categories that we’ll go deeper into. Things like wine. We’ll focus on these categories, but we’re very different than [traditional shopping comparison engines] as we don’t have categories like Shopping.com does. We’re much more like Google, a general purpose engine, trying to make everything work very well.
“In the end, we’ll keep making the product better. We can add all types of features, but that won’t matter if the prices are wrong. Our data extraction has to continue to improve.”
“We’ll expand internationally this fall. The data feed model is a bigger problem in smaller countries. International was huge part of DoubleClick’s success, and we’ll be in a lot of countries next year. We’ll be indexing everything that’s available, but if you’re in the US, we won’t show you results for France, for instance. However, if you search for 1920s tiffany lamp and it’s only available in France, we’ll show it because there’s an incremental benefit to the user. ”
Can you talk about the Wikis? Where do the articles come from?
“The Wikis are written by us, friends, or other in house people. We have reached out to people and we have in house editors. Compared to Wikipedia, we’re exerting more editorial control. Someone looks at every change because the community is not yet big enough for self policing. We’ll maintain a small team of writers and editors.”
Business model – How do you plan to make money? You say you will accept advertising in the future, will it be keyword based? Are you currently an Amazon affiliate?
“We’ll have keyword ads on our pages on the right hand side and bottom of the page. We’ll use other people’s ads to start, and when we’re larger, we’ll see about doing our own. [Obviously Kevin has some experience running ads on sites.]
“If you look at a page on some of the leading shopping comparison engines, [we think] they make approximately $70 per 1000 impressions, made up of $40 per 1000 for the product search results and $30 per 1000 for the AdSense results. We’ll get less revenue per page [as we don’t get paid for our ‘organic’ results], but the cost per page will be dramatically less. A leading engine might have 150 people working on all the touch points – sales, customer service, engineering, etc – with the stores to manage those product results. We only have engineers. And with a handful of engineers, we can create a product with better results, more features, more stores, etc. They might have 50 engineers, we have 6.”
“The [current breed] of shopping comparison engines don’t have this business model, because 5 years ago, they didn’t have the choice [of AdSense or Yahoo! Publisher Network (YPN) ads] so they had to sell advertising in the form of data feeds.”
If you have ‘everything’ indexed, why will people click on the Google AdSense or YPN ads?
“Well, Google has ‘everything’ indexed, but there’s still $10B worth of clicks on Google. People never feel like they click on ads, but they do. ”
“The AdSense ads are hand tuned. An advertiser manually said for this particular keyword, this is what I think the user is looking for and this is the page that I’m going to put up. The AdSense ads can be pinpoint accurate.”
“In the end, the revenue we will generate on the ads compared to what our competitors generate will be less because we have a high degree of accuracy. However, the cost structure [of ShopWiki] is pretty light, so we can handle a little less revenue per page. We will create the right product for consumers, and that will make the difference.”
“Our competitors optimize around revenue per page and constantly tweak the pages to squeeze out a little more revenue. When they pay $0.30 to get someone on the site, they have to make sure they make money on that person.”
“We’re focused on the user experience.”
ShopWiki Launch Press Release – April 19, 2006
Dulance Acquired by Google – April 12, 2006
InterActive Corporation’s Pronto – April 4, 2006
ShopWiki – CTO & Founder, Eliot Horowitz – March 30, 2006
Dulance – We Hardly Knew You – March 29, 2006
Dulance – The Long Tail – July 20, 2006