Click Here for Details
Wine Quality Scores - Why Do They Vary?
....Email this article to a friend....
Wine magazines that conduct blind tastings using experienced wine tasters offer valuable and
unbiased information to their readers as to the quality of wines. However, the value of
such ratings are often questioned, as it is common for different ratings to be given to the
same wine across different wine tastings. So why does this occur?
We can answer this by first exploring the circumstances under which the quality ratings given
would always be the same. If we had a pool of tasters who perceived the sensory quality of the
wines identically, interpreted quality parameters identically, and used exactly the same criteria
for assessing quality, then yes, we would achieve identical quality ratings. However, quality is
not absolute. It varies according to the context in which its is being viewed, the criteria being
applied when viewing it, and to some extent by who is doing the viewing.
A typical quality assessment panel consists of a three of four experienced wine tasters. The individual
tasters first assess the quality of a group of wines independently (and always blind) and then briefly
discuss them before reaching a consensus. The anonymity of the wines is maintained until a decision is reached.
The evaluation of the wines is partly dependent on the combined performance of the panel's sensory
equipment. The actual degree of taster variation is not widely appreciated. Sensory scientists use
a rule of thumb that typically the most sensitive five percent of the population will be 250 times more
sensitive to an odour than the bottom five percent. Some tasters have up to 30 times the number of
taste receptors than others, and the volume and composition of saliva produced, which affects the way
in which were perceive tastes and astringency varies even more. What this means is that some panels
may decide on the existence of a fault, or see a wine as being out of balance, while another panel
may not. Discussion by tasters assists in evening out many of these differences, but this act itself
can introduce variation through the panel dynamics that arise.
The most important factor in explaining the paradox is that of the use of different quality criteria.
This is best explained by example. Assume that we have a full-bodied Cabernet Sauvignon that has only
moderate flavour levels, but distinct varietal character, and a typical level of acidity for a Cabernet
(but a higher level than the average red table wine). Assume that the wine was tasted in a line up of
straight Cabernet varietals. In this context, the judges would appreciate its varietal expression and
although being 'highish' in acidity, it is considered to be within the acceptable and expected limits
for a Cabernet . Using these criteria, the judges would rate the wine highly.
Assume that the same judges are invited back, but this time to assess a set of wines as full bodied dry
reds. In this context the wine described above would be viewed as being only moderately flavoured with
an inappropriate acid balance. As such, it would be rated lower than previously. Nothing sinister here, only that there was an appropriate change in perspective the part of the judges.
To compound matters all wine tasters, no matter how experienced have slightly different interpretations as to what constitutes quality. Sure, all judges would agree on the general parameters that underlie quality: aroma and flavour intensity, complexity, balance, structure, attractive mouthfeel and length. However, judges weigh the importance of these differently depending on their experience, training and personal philosophy. Therefore, even with everything else being equal, different panels of judges will not always exactly concur in their opinion as to the relative quality of a set of wines.
Another factor that affects quality perception and ratings across different occasions concerns relativity between the wines being assessed. As humans are excellent at comparing things, but are far less able to relate things in an absolute sense. As a result, we find it difficult to assess the quality of a wine given in a line-up completely independently of the wines around it. For example, there is a tendency to rate a good wine presented among a a set of poor wines a little higher than if it were assessed among a
set of other good wines. Individualistic wines tend to be rated slightly higher in the case where sets
of very good, but uniform wines are presented. Furthermore, subtle wines tend to be underrated if they
are tasted among bigger tannic wines particularly when palate fatigue has set in. There are other
examples of these types of context effects, so it is no wonder that the same wine in a different
line-up is rated differently at times.
So what is the take home message? If a wine is a consistent star rater, being seen positively by a
variety of judges using different quality criteria among different lineups, then in all likelihood it
is an excellent wine. Matched with a gold medal or two from respected wine shows, it is difficult to