For most people, this post is too long and too boring to read, so I’ll give you the quick rundown. I refuse to do the ‘best’ shopping comparison engine post (for consumers) because that would include comparing the results of multiple product searches in multiple categories across about 10 engines. Below I do this exercise for 1 product in 1 category across 5 shopping comparison engines. And it was a hellish experience. While I appreciate the efforts of Consumer Reports and others who do a yearly ‘best’ article, I don’t think these publications are digging deep enough.
Just so you don’t have to read through this entire post, here are some bullet points which identify problems and suggest improvements.
In general (yes, I hate generalizations, but they make life easier):
-shopping comparison engines do a poor job of displaying accurate Shipping & Handling (S&H) and tax information. In some cases, this leads to price discrepancies of around 15% between the price a shopping comparison engine lists and the actual merchant price
-shopping comparison engines should display rebate/coupon/deal information in a consistent manner. All prices displayed should be before rebates/coupons/deals or after rebates/coupons/deals resulting in an apples to apples comparisons instead of jumbled apples to blackberries comparisons
-merchants are either purposely posting incorrect data on the shopping comparison engines or are not paying enough attention to the channel. Either way, this leads to a poor user experience.
-if a shopping engine has my zip code, the price range displayed (above the comparison listings) should reflect this and include tax and S&H costs
-if the merchant is Amazon, but it’s going to be shipped by a third party, the user should be notified up front by the shopping comparison engine
-if more than one merchant lists the same rebate, the shopping comparison engine should pick up on this and give the user a special note (outside of the price comparison) that there may be a deal available for the product
Every shopping comparison engine should proactively address the issues I bring up in this post. Seriously. It’s embarrasing that after 11 years of comparison shopping there are so many errors in the data. I’ll be the first to admit that I picked 1 product out of the 10s of millions listed on the engines, so for the statisticly minded reading this post, feel free to move on right now. For those of you still with me, the errors I uncovered led to an extremely frustrating experience that I’m sure would make the average consumer think twice about using these services (assuming they know where they are in the first place). As the engines try to figure out how to lessen their reliance on Google/Yahoo, they have to pay attention to the user experience. If 30m people a month visit these engines and the information is this spotty, imagine the potential for a shopping search engine with clean, correct, and comprehensive data.
And the onus to improve doesn’t just fall on the shopping engines, but also on the merchants who are providing the data. There is an initiative to create a standardized data feed for the shopping comparison engines, but this initiative will not fix the data quality problem. In the search I conducted, one merchant listed S&H of $1.99 on Shopping.com and Shopzilla, but the S&H cost ended up being a lot more. I have no reason to believe this is a nefarious strategy (list a low S&H price to get a high click-through rate), but it reflects poorly on both the shopping comparison engines and the merchant.
Now to the ‘way too long’ post. I apologize for the my poor editing, but it was just time to move on.
Buy.com seems to have the best price on the D-Link MediaLounge DSM-320 Wireless Media Player at $149.10 before a $20 mail in rebate available from D-Link. Buy.com links to the D-Link rebate from the D-Link DSM-320 product page to make it easy for the consumer to take advantage of the deal (although Buy.com might want to prominently display it throughout the checkout process). Add in $11.42 shipping to my address in CA and no tax and the total comes to $160.52. Take off $20 for the rebate, and my total cost is $140.52.
On the official D-Link Shop, the base price is $159.99, tax is $13.20, and S&H is $14.01 for a total of $187.20. Take off $20 for the rebate, and my total cost is $167.20.
D-Link Price Before Rebate: $187.20
Buy.com Price Before Rebate: 160.52
Total Savings: $26.68 or 14.25%
I knew about the $20 rebate because of Marc Mezzacca’s mashup between Shopping.com’s API, Amazon Web Services, and a database of coupon/rebate information pulled from a number of sources. On his site, SecretPrices.com, he prominently shows the $20 rebate. Although he got the S&H information wrong a little further down on the page – Buy.com didn’t offer me free shipping – because he listed the $20 rebate, Marc still pointed me in the right direction. If you take a close look at the DSM320 page on SecretPrices, you’ll also see that Marc clearly lists an ‘After Rebate’ column. Keep this in mind as you look at the way most engines display the rebate.
The point of this post was supposed to be about Marc’s site (the idea of adding deals and coupons to product comparison pages is not at all new, but unfortunately, not many shopping comparison engines have implemented this effectively), but as I checked out a number of shopping comparison engines to see how they stacked up to this one search on SecretPrices.com, I became extremely frustrated as I realized that after 11 years, the shopping comparison engines have a long way to go in providing a positive, relevant, comprehensive, and useful user experience.
Check out these search results for a D-Link DSM 320. You might get different results depending on where you live and current offers, so I’ve included screenshots below documenting my experience. The screenshots were taken over the last week.
-Shopping.com’s search results tell me that they have prices for the product ranging from $140 – $300 at 22 stores. The first page of results, though, shows only 4 merchants with a low price of $153.53. This is a fairly poor user experience from the start. I obviously want to see that $140 low price and Shopping.com is forcing me to take an extra step to find it. Actually 2 extra steps. I first click the ‘Compare all 22 store offers…’ and then click the ‘Total Price’ link to sort by price. Ok, I don’t see the $140 offer. The best total price according to Shopping.com seems to be from eCOST.com at $153.53 (base of $139.99, with tax of $11.65, S&H of $1.99). Amazon.com is next at $158.99 (base of $158.99, with tax of $0.00, S&H of $0.00). The third listing is from Buy.com at $161.40 (base of $149.10, with tax of $12.30, S&H of $0.00).
Seems pretty straight forward so far, but there are a lot of issues already. Upon closer inspection, the best total price seems to be from Buy.com. The eCOST.com listing has a note which says ‘After $20 Rebate’ while the Buy.com listing has a note which says ‘BEFORE $20 REBATE’ (yes, it was all in caps, which probably shouldn’t have been allowed). Because of this discrepancy in the way rebates are included in the listings, I’m not too happy with my Shopping.com experience.
Furthermore, when I click through to eCOST, the base price I see is $139.99, but the price jumps to $159.99 as soon as you add the product to the cart (as you have to send in a form to get the rebate). Tax turns out to be $13.20, a bit more than the $11.65 stated on Shopping.com. More importantly, the $1.99 S&H is completely wrong. On eCOST, the S&H turns out to be $11.89. Ouch. eCOST does offer free shipping, but it’s through a special club which I have to join for $39.95. I’m really not sure where the $1.99 comes from. Total cost is $185.08 or $165.08 after the $20 rebate compared to the Shopping.com price of $153.53. That’s a $11.55 or 7% difference.
Now to Amazon. When I click through, I’m told the product is being shipped and sold by J&R Music and Computer World. Not a big deal (Amazon is ‘outsourcing’ more and more as it’s a high margin business), but it would have been nice to know this up front. I add the product to my cart, login to Amazon, choose the address, choose standard shipping (3 to 7 business days), and at the very end, I find out that S&H is $16.08, not free (I think this is because it’s a marketplace sale, not an Amazon sale) as Shopping.com had stated. Total cost for the D-Link DSM-320 is $175.07. This compares to the $158.99 displayed on Shopping.com. Also, Amazon/J&R made no mention of the $20 rebate.
Now to Buy.com. Shopping had the S&H and tax wrong on this one, but both numbers pretty much cancel each other out. S&H totals $11.42 as oppsed to being free, but there is no tax as opposed to the $12.30 Shopping.com thought I’d be charged. Total cost at Buy.com is $160.52,. This compares to $161.40 displayed on Shopping.com. Also, Buy.com made the rebate information pretty clear, so total price will eventually be $140.52.
-Shopzilla’s results are very similar to Shopping.com’s – although I didn’t have to click twice to see a full list of merchants. Shopzilla has 33 merchants selling the D-Link DSM 320 with prices ranging from $140 to $329. Just like Shopping.com, the low price of $140 reflects a base price and not the total price including S&H and tax.
Once again, eCOST offers the lowest total price at $153.53 (base price of $139.99, tax of $11.55, and S&H of $1.99). Immediately, I know this information is misleading/wrong for a variety of reasons. First, the base price is actually $159.99 and comes down to $139.99 after a $20 mail in rebate which Shopzilla didn’t mention. Second, tax is actually $13.20, not $11.55. Third, S&H is $11.89, not free. Shopzilla got this information completely wrong just as Shopping.com did. In the end, my total price is $185.08 before the $20 rebate or $165.08 after the rebate compared to the price of $153.53 displayed on Shopzilla.
The second lowest price according to Shopzilla is offered by PCNation which sells the DSM 320 for $161.10 (no tax, free shipping). Happily surprised to see this is the actual price listed on PCNation, but my hopes for an error free experience are soon dashed after I enter my shipping/billing info in CA as I’m told “PCNation is currently unable to ship orders from some of our distribution centers to addresses within the States of California and Massachusetts without adding a handling fee related to the local state sales tax.” Ouch. There are 2 choices for me at this point, either agree to a $18.00 warehouse transfer fee or a handling fee related to the sales tax of $13.00 (this option adds 2 days to the delivery time). I choose option 2 and my total cost turns out to be $174.10 compared to the $161.10 displayed on Shopzilla.
Third choice on Shopzilla is Buy.com with a price of $161.40 (base price of $149.10, tax of $12.30, and Free Shipping). The note in the listing says ‘Price before rebate’. Once again, as with Shopping.com, the tax is incorrect and the S&H is incorrect. However, as seen above, these two details cancel each other out. Total cost at Buy.com is $160.52,. This compares to $161.40 displayed on Shopzilla. Also, Buy.com made the rebate information pretty clear, so total price will eventually be $141.40.
-PriceGrabber’s search results page tells me they have 36 sellers of the D-Link DSM 320 with prices ranging from $149.10 – $201.48. PriceGrabber also points out the $20 rebate form directly from its site as opposed to depending on the merchants for this information…a nice implementation.
The lowest priced merchant is Buy.com with a ‘BottomLinePrice’ of $160.66 (base of $149.10, tax of $11.56, and free S&H). We know the base is correct (and doesn’t reflect the rebate which Buy.com displays on its site or the rebate which PriceGrabber displays), but again the tax and S&H information is incorrect.
Electronics Plus is the second lowest priced merchant with a ‘BottomLinePrice’ of $168.25 (base of $153.50, no tax, S&H of $14.75). Clicking through to the merchant reveals its a PriceGrabber Storefront listing. That’s fine. S&H seem to be listed correctly. I run into my first problem when I enter different S&H and Billing addresses as I’m quickly told “The seller declines transactions if billing and shipping addresses are different.” Once I settle this issue, it turns out there really isn’t any tax and PriceGrabber returned the first completely correct listing. The 8th time is a charm. Total price is $168.25 which is exactly what PriceGrabber said it would be.
The third listing is for TheTwisterGroup.com with a ‘BottomLinePrice’ of $168.53 (base of $155.12, no tax, S&H of $13.41). Everthing works out with TheTwisterGroup.com as the BottomLinePrice listed on PriceGrabber equals the final price on the website.
Because PriceGrabber seems to be on target, I check out the listings for PCNation and eCOST. PC Nation price on PriceGrabber is $174.39 compared to the real price of $174.10 (the tax information is a little off, but a $0.29 error is not that bad). eCOST has a price on PriceGrabber of $192.77 compared to the real cost of $185.08 (before the $20 rebate). Turns out the eCOST base price is correct while the tax is listed incorrectly at $13.94 (should be $13.20) and S&H is listed incorrectly at $18.84 (should be $11.89). In other words, the BottomLinePrice on PriceGrabber for eCOST is about $7 too high.
-NexTag returns 15 sellers offering the D-Link DSM 320 from $129.10. Ok, the $129.10 turns out to be Buy.com’s price after a $20 rebate, but without the S&H or tax information.
NexTag clearly displays – in Gold – that there’s a $20 rebate involved with the Buy.com price. However, NexTag, like everyone else, lists the tax and S&H for Buy.com incorrectly. Tax is listed as $10.65 when there isn’t any tax and S&H is listed as free when in reality it’s $11.42..again cancelling each other out. As we’ve seen before, my final price on Buy.com after rebate will be $140.52 and NexTag’s TruePrice is listed as $139.75.
The second listing on NexTag in order of lowest price is for Amazon.com at $158.99 ($158.99 with no tax, no S&H). When I click through, it turns out the price is $165.69. I’m directed to a different Amazon Marketplace (NYC Electronics) than the one I found through Shopping.com and Shopzilla (J&R Computer World). While the $158.99 price is available through J&R if I scan the page carefully, I don’t like that NexTag said the price was $158.99 and then pointed me to a page with a price of $165.69. Second problem is the S&H. Once again, a shopping comparison engine is telling me there’s no S&H while Amazon is telling me a different story. Third problem is that NexTag displays this product as ‘Refurbished’ while Amazon/NYC Electronics says it’s new. Total price turns out to be $178.20, not the $158.99 NexTag displayed.
The third listing on NexTag is PCNation with a True Price of $161.10 (base price of $161.10, free S&H, no tax). We’ve seen this one before and know it’s a mistage. The base price is correct, but PCNation (if you recall) has trouble with Californians and charges me an extra $13 to live in the state. Total price is $173.10 as opposed to the $161.10 which NexTag displays.
Next up is Yahoo! Shopping which has 25 merchants selling the D-Link DSM 320 with prices from $139.99 to $242. As we’ve seen on other comparison engines, the $139.99 price is misleading as this is actually just the base price and does not reflect tax or S&H.
The lowest priced merchant on Yahoo! Shopping is GainServer with a total price of $157.75 (base price of $152.59, no tax, $5.16 S&H). Turns out Yahoo! can’t get these prices right either. Base price is correct, but tax is $12.59 (not free) and S&H is $11.18 (not $5.16). Total price on GainServer is $176.36, not $157.75 as Yahoo! Shopping reported.
The second lowest priced merchant on Yahoo! Shopping is TigerDirect with a total price of $157.98 (base price of $149.99 (after rebate), no tax, and $7.99 in S&H). Yahoo! Shopping clearly pointed out this price included the rebate through the use of an asterisk (*). Turns out this is correct.
The third merchant is TheTwisterGroup with a total price of $168.53 (base price of $155.12, no tax, S&H of $13.41). We know from a previous experience (see PriceGrabber above) that this is correct.
Since Yahoo! Shopping fared well with 2 results, I took a look at some other merchants that other shopping comparison engines had trouble with like eCOST and PCNation. Yahoo! Shopping, like everyone else, has the S&H wrong for eCOST, and I’m told to check the store for the tax…I guess that’s better than getting it wrong. As for PCNation, Yahoo! Shopping lists the total price as $173.98 (base price of $161.60, tax of $12.88, and Free Shipping). This is extremely close to the actual price $174.10 – Yahoo! was just a bit off on the tax.
Thanks for doing this. I periodically generate this kind of comparison for travel sites, and it is indeed as tough as you indicate to drill all the way down. You are also correct in observing that publications and commenters currently don’t dig deep enough.
I think most comparison engines today generate what is primarily a ‘faux’ price comparison experience – results which look like comprehensive price listings, but as soon as you get ‘under the hood’ you realize they are far from that.
These companies are trading on precisely the difficulty of the task as displayed in your post. The egregious example which comes to mind in travel is Kelkoo, which last time I looked at it consisted on most searches of no more than three or four online travel agents stitched together, to simulate a comprehensive price search (the online travel agents of course also try to make their offers look like the results of a comprehensive price search).
The ‘faux’ version of the comparison shopping is of course working commercially very well at present – Kelkoo sold for $580 million because it was highly profitable.
In truth all of these players are exposed to the arrival of true price transparency. It’s ironic that shopping comparison engines as we know them today are as exposed as the businesses they cover, and perhaps much more so, to this development.
“There is an initiative to create a standardized data feed for the shopping comparison engines…”
Can you point me to this?
(And, as always, thanks for the great post!)
Sounds like the same merchants had Price errors across different sites. Could it be that they are just not feeding the right information to comparison shopping sites?
Ofer,
A group of Retailers, Solution Providers, and Shopping sites have started the Online Retail Datafeed Standards (ORDS) group under the Association for Retail Technology Standards (ARTS) which is a part of the National Retailing Federation (NRF). Key participants are Yahoo!, Microsoft, AOL, IBM, etc.
The mission is to create an XML standard for product feeds and to market this standard to the online community.
It seems to me that some of these merchants (eCost, for example) may be intentionally manipulating the data they give to these sites to better catch the shopper’s attention (as they did with you in your comparison). I think the onus should also be on the merchants to provide honest data to their customers, whether they’re providing that data on their own site or via a comparison shopping site or search engine.
Thanks, Greg!
I am just glad the word “nefarious” is being used. Nice work Brian, great post.
Cheers,
Nathan
I have wasted hours using shopping comparison search facilities. Just because one website/retailer seems to be the top in best value for a specific product in a league table, this can change and become more expensive, as you have said, when you take into account other costs such as shipping.