01438 741177         thewinesociety.com

Measuring quality


#1

I’ve been listening to Tick Box Art on BBC Radio 4, about whether and how we should measure the quality of art. I couldn’t help but think how much of what is discussed could be (and perhaps should be!) applied to wine.

They raise the important questions of liking vs quality, knowledge/learnedness vs instinctive appreciation. Soweto Kinch, jazz musician, recounted a clever analogy of a non-jazz listener learning to appreciate jazz - one that could well be applied to someone just beginning to venture into the world of wine. Learning Russian indeed… (I won’t repeat it here - listen to the program, it’s worth it.)

But what I found most compelling was the assertion that ‘it is not possible to measure quality in an art work. Whenever we ourselves engage with an art work, whenever we talk about it with friends or write about it or reflect upon it, we never talk about numbers. We never talk about metrics, we never try and measure it. And that’s because it’s absolutely impossible. It’s entirely the wrong way to go about evaluating art work…’

Since the early eighties, the wine world has become more and more obsessed with scores. Personally, I feel that what applies to a piece of music, a painting, a sculpture, dance, could and should apply to wine. I don’t for one minute feel that the role of the critic is redundant (indeed, this program also points out the importance of the role of the critic), but today’s wine critics seem unable to tear themselves away from the score card. It reduces the value of wine to a number, and causes plenty of confusion (is a 16 point Chambertin Grand Cru inferior to a 16.5 point Bourgogne Rouge? Or are they being marked against their own class rather than in a giant pool of wines? And if the latter, should we ever compare a Burgundy Pinot to a New Zealand Pinot?). And it conclusively distracts from what I think is the primary point of reviewing a wine: does it taste good?

Anyone feel that there is a good argument for scoring wine?


Rousseau 2015
#2

WOW - really big question … and I need to properly think about the answer and check out the programme, naturally.

In summary, my answer will be an emphatic NO. It is the reason I have never agreed to do any judging. I cannot get my head around the application of a score to a sensory experience.

However, the thoughtful question deserves a more thoughtful answer, so I shall ponder (over a lovely glass of wine - including a tasty half bottle of Bollinger my wife bought me)


#3

Well in gymnastics you have points for difficulty and execution. Maybe it takes the apellation as attempted difficulty and execution as the result? Not sure about new world? Maybe price as one and quality as the other?


#4

There are a few assertions here that I’m not sure I agree with. Leaving artistic creations to one side (I hope we can measure their influence at least) and not having listened yet to the documentary, I do find scores useful provided they are in context.

The first context is other scores from the same tasting or producer. I do want to compare wines and to see the critic make some attempt to guide me in that comparative judgment. Why should we be afraid of comparing things and finding some more winning than others.

Secondly experts are very important - not every palate has the same training and I hope there are sufficient objective criteria that can be applied to tastings to make it possible to quantify them. And that might not be just whether it tastes good but also whether the winemaker working within their own practical limitations is able to express fully what they intend to. Can’t we quantify that and don’t we need experts to do so?

Thirdly, a score is a useful proxy for affordability. Most wines scoring > 17+ are too expensive for me, and I assume the vast majority score 15.5-17. So the scores often guide me towards what is within my budget (or should be). (Certainly seems true for Bordeaux/Rhone/Burgundy).

Having said all that what normally captures my attention and makes me want to buy something however is the actual tasting note (preferably brief and brilliant).


#5

@cgoldin do you think as most wines score in that narrow band by proxy 15 becomes 1 and 17 becomes 10? Seems a silly scale as does the 100 as most of the wines in lidl score nearly 90 and some top bordeaux only get a few % more…i’m confused.


#6

I think with the Robert Parker scale, anything below 59 is unacceptable so you could almost reset the Parker points and start with about 70 points as 0.

https://www.robertparker.com/ratings

I think points can be useful as if its from the same taster they can be compared against each other, alas due to the overall improvement in quality there are now some cheaper wines moving into the higher points (which is good for consumers!) but means that most wines are now in the top set of points whereas previously there would have been more of a range. For me personally I don’t really put much behind an experts points, they can be useful as a guide but not the be all and end all, I have tried a few vintages of the Penfolds Grange wines which I found were well made wines but I wouldn’t say outstanding whereas they usually receive very high points scores.


#7

Part of the problem is that wine critics, like, for example Jamie Goode, only publish scores on good wines. So in most cases their base score is around 88-90 and the scale could be smaller still.


#8

Couldn’t agree more, Richard. Many wine critics score within a very small scale. The 20-point scale really only goes from 15 to 19, 19 being super-rare and 20 being almost unobtainable. So 15-18 being more realistic. We could call it a 5-point scale. And the 100-point scale is mostly populated from 87-98 - we could call it an 11-point scale.

But my biggest concern is that scores imply that wine can be measured absolutely. And it cannot, because it is deeply subjective.

There is a fascinating book out called Neuroenology, by Professor Gordon Shepherd, head of Neuroscience at Yale School of Medicine, who has spent the last 60 years studying how our senses and brain perceive the world around us. This book focuses on how we taste wine. When you begin to scratch the surface of the variables - beginning with the differing shapes of our nasal cavities, the make-up of our saliva (which is affected by DNA, gender, hormone cycles, health, nutrition, hydration and time of day), the number of aroma receptor cells we each have, our own personal experiences, sensitivities and preferences - it is absolutely stunning that we ever agree on anything at all. And this doesn’t take into account the accepted changeability of wine, which is affected by serving temperature, serving vessel, time open, humidity, age and storage, to name but a few factors.

I am very aware, in my own job, how the reviews of one wine can differ wildly from person to person, and day to day -
and these are all experts reviewing the same wine. I can’t help thinking that scores for wines is a wee bit like tying down the wind. I spend most of my everyday submerged in wine scores, and I wonder if it’s not an audacity to think that we can place an authoritative and conclusive score on a wine.


#9

OK, so I have not finished formulating this thought, but sharing to start the conversation

Scores on wines are NOT a reflection of the quality or the intrinsic value of a wine. They are, in fact, a guide to the consumer allocation of scarce resources.

Wine is NOT like art, because art can be consumed many times by an unlimited number of people (though only a few can ‘own’ it, whatever that means).

Wine is to Art as Gallery Visits are to Tastings

Imagine a world where art is only possible at very specific times and places. There are a finite number of tickets, after which the art will no longer be displayed. You do not need to pay for the art itself, only access to it.

You would not just need an indication of the ‘quality’ of the art, but of the whole experience of visiting; the queuing, the lighting, the comfort, the exclusivity and how close you can get to the art, in order to decide where and when to spend your limited resources to get a ticket. In the same way that theatre events, or galleries, get star ratings in newspapers.

This does not fundamentally affect the issue of rating scales and limited value of 100 or 20 point scales, but it does give some justification of the use of these scores to ‘rate’ wines. We are only buying ‘windows into the world of wine’ and we there are limited tickets, so which ones should we be buying so that we can THEN make our own judgements about the quality of both the wine/art and the experience

Food for thought?


#10

So there should be unlimited free wine then? :smiley:


#11

So there should be unlimited free wine then? :smiley:

Well kind of… :stuck_out_tongue:


#12

My assumption is that if tasted in the same session by the same taster, then in the opinion of that taster probably YES. Even in this case there are of course uncertainties… In any other case god (though would hate to bring them into this argument) only knows.


#13

Should it be horses for courses perhaps?

Absolutely spot on and definitely food for thought. Just as some, not all, pictures and sculptures etc are works of art, so too are some wine, some Michelin standard food and some perfume works of art too. Albeit, these latter works of art are also made to be consumed, in there own way of course, which puts them on a completely different level altogether from all other art…

if it was just simply pictures, for instance, how would you rank a print, of say the Mona Lisa and compare this to, say an original Picasso’ Cubist painting? After all, they are both pictures aren’t they? Now that’s exactly how I view the wine world too. There are oceans of different mass produced wines out there, branded wines that never change; are always available; the quality is always consistent and they are very, very popular and this keeps the cost extremely low. Conversely, at the other extreme, there are also extremely high quality, virtually hand made, one off wines, wines that are so rare and some that are not even made every year. But because these cult, “artisan” wines are so “good”, people want more than is available, so the price is extremely high. So, why are we still trying to compare everything, with everything else, (or should that be how can we, sensibly)?


#14

Unless they taste from the same bottle at the same time, they are not the same wine.

There can and often are differences between bottles of the ‘same’ wine, sometime slight, other times major.

At our local wine tasting club we’ve opened 5 bottles from a 6-pack of wine, tasted each one for corkiness and find frequently differences between the bottles.

If you regularly buy the same wine, as I do the Society Montepulciano, look at how the lot number changes. On the Montepulciano I think its the month and year the wine is bottled. Later bottlings of the same vintage will have been in the tank for longer, older bottlings of the same vintage will have had more bottle aging.

Same wine?


#15

It seems I was more prophetic than I realised


#16

Prophetic, indeed @robert_mcintosh! :+1:
It also brings to mind the other similarity between consuming wine and consuming (if that’s not too cannibalistic a term) art. What might seem an extortionate price to view art for one person, another would find reasonable, and a reflection of their appreciation of this particular art/ist. Similarly, one person finds it reasonable to pay, say, £80 for a bottle of a wine/producer they love and appreciate, whereas another person would find it bonkers. Wallets and tastes differ, and we all prioritise our purchases based on both. What I find remains true for me, whether in art or wine, is that beauty is in the eye of the beholder (and the specific palate of the specific drinker).


#17

I have said my piece on this a couple of times in other threads so wont go there.
Decanter has had a flurry of letters condemning the change in the way they will judge wines, in essence although they will taste blind the wines will be in set price bands, quite rightly the letters point out that critics as with anyone will have a pre concieved view about the quality of wine with a price band, (this has been proved in tests elsewhere) and therefore those cheaper wines that would score well in the past as there was no marker for them will now be judged simply as cheaper, inferior ? wines.
The old method often threw up cheaper wines with high scores and excellent tasting notes, that is now not going to happen as the wines are seperated by price, the change was questioned for several reasons, one being that were Decanter embarrased when a cheaper wine topped many expensive ones and did they feel it undermined their tasting skills, and why shouldn’t cheaper wines be tasted alongside their expensive peers, if they are as described they have nothing to fear, but of course they do, the response from Decanter was not convincing citing as there main reason that cheaper wines under the old method only rarely made the top scoreboard and they felt it was unfair ?
Yet this practise has been used happily by the magazine for decades with no complaints and many happy wallet saving surprises.
The elite wineries don’t like that sort of surprise, hence you don’t see them in wine competitions, yes they don’t need to compete as they have sold their wine allocations well in advance and are not going to take a chance with the hoi polloi, those rare occasions when tasting blind and an imposter has got in the mix and done very well are not something they would volunteer for.
Scoring , whatever method is used is useful if read alongside notes and then compared with other tasters including “amateurs” who often do not vary that much anyway apart from having the bottle to say bad things about a wine that pros do not dare to utter, after all before the power of Parker critics who did not toe the line were in some cases barred from that winery, which was inclined to shut down any non glowing reports from others.
For me and many others the change at Decanter is viewed as retrograde and the reasons are not persuasive.


#18

I wholeheartedly agree, @cerberus ! :+1:
I, too, read about this change, and felt a mixture of surprise and dismay. Their response to the criticism certainly confirmed my more cynical feelings about such panel tastings and judgments in general.


#19

If you assume impartiality in Decanter reports you’re starting with the wrong premise. It is an advertising driven publication and there are financial interests in promoting producers from the month’s selected region/country. This is also true about other wine critics and publications. Read all critics with this knowledge.
The most informative views are those of knowledgeable amateurs who have no financial interest in what they publish.


#20

Agree with cerberus and wineyg. The arguments given by Decanter for the change in policy were embarrassingly thin and not to their credit.