On January the 19th we released the comprehensive analysis of the Metacritic website regarding their Games data. As a way to shed some light on the findings reported in that paper, I write this annotated blog. It will save you from having to read the 30 pages.
Introduction
The Metacritic website has a number of important flaws according to the Wikipedia page:
Metacritic:
• Converts each review into a percentage the site decides for itself
• Manually assesses the tone of reviews that have no explicit score and then assigns a quantitative score
• Refuses to reveal what weights are applied to which publications
Now, assuming the above is correct, this would make the data shown at Metacritic highly biased and far from the truth. Statistics dictate that you cannot apply scores at your own leisure based on subjective perception or using unvalidated methods and then call that a sophisticated meta-analysis. Therefore, it is time someone took a closer look at the data at Metacritic, in a descriptive way, to explore the data and see what comes up.
(more…)