Regards.

Is it something where you can say (like Avinash Kaushik suggests doing), well, I don’t have full confidence in the data, and I don’t even have 50% confidence —

But maybe I have 10% confidence, and I can still make a (small) decision based on that.

from the post:

You mentioned that Compete will attempt to verify and reconcile its numbers against what many of the larger sites listed are reporting. Can you talk about that process and what’s entailed? This seems like something that might give Compete a real competitive edge - do you publicize when these connections are complete? How many have you done over the past year?

The process is somewhat ad hoc. Essentially when a company or site owner contacts us about discrepancies between their data and our own or even between our data and another third party source we look at the variation to determine if the gap is substantial or not. If it does look substantial we dive into a couple of areas. First we look across our 10+ different sources of data to see if there are any panel bias issues. For instance does a certain type of panel have abnormally low statistics for the given site. If we do see this than we can apply an adjustment to our normalization process that accounts for this. If this does not address the issue we then take a look at external factors that might address the issue.

In most cases if we can’t identify a normalization adjustment we can generally find an external factor that at least rationalizes the issue. At the end of the process there is really no “public” announcement or anything of that kind. Over the past year or so we have probably gone through this process for at least 50 domains. The bottom line is that we don’t simply change the numbers to match the sites internal numbers. We look for a logic explanation as to why our projection and normalization process is generating different values and adjust as necessary.

i recommend reading the interview. learnt a lot.

///If you compare our data and the SELand data to what Compete, Quantcast or Alexa are reporting, you can see how tragically inaccurate those services are. Sadly, that’s no anomally. Everytime I get access to a client’s visit data, I’m always curious to check the three and have not once found accuracy, even on a relative basis. Third party traffic metrics still have a very, very long way to go

http://www.seomoz.org/blog/seomozs-stats-for-2007