Does being underrated have statistical meaning? (I don't have an answer.)
While I was reading Eater's article on "NYC's Most Underrated Restaurants" (link), a thought keeps troubling me. Here, the editors conducted a poll of the readers to discover which restaurants are underrated.
That sounds innocuous enough but something is amiss. A poll measures the average opinion. An underrated restaurant is one in which the average opinion is purportedly wrong. Something about using an instrument for discovering the average opinion to disprove the average opinion doesn't sit right.
***
So I set about thinking how one might mathematically describe the idea of being underrated.
The most obvious choice seems to be a gap between an intrinsic rating and the average opinion of the rating. The intrinsic rating can never be measured, and we use polls to estimate its value (really, a set of values). The underrated restaurant receives a poll rating that is below its intrinsic rating.
Something doesn't sound right about that too. The intrinsic rating simply doesn't exist. There is no higher being to tell us which restaurant is better than which other restaurant.
So, perhaps we can compare a poll rating with a "population" rating. The population rating is defined as what the rating would have been if everyone who cares to rate restaurants is polled. But now the gap between the two ratings is just the error in the sample. If the sample size of the poll is large enough, the concept of underrated (or overrated) would dissolve.
(One could argue that the population rating is the intrinsic rating because if one must find a value for the intrinsic rating, a good choice would be the population rating.)
***
Maybe underrated is a function of a subset of the population. An object is considered underrated by a subset of a population if the rating by that subset is substantially lower than the population rating. Put simply, it's just a cliche of people, like readers of Eater, voicing their dissent against the average opinion.
But one can find such a subset in any sample since there must be people who rate a given restaurant lower than average. Can there ever be a consensus of an underrated something?
Let me know if you have any thoughts on this.