These Olive Varieties Earn the Highest Scores from Judges

There are a few rea­sons we don’t reveal the scores our judges attribute to entries in the NYIOOC World Olive Oil Competition, but it boils down to the fact that they should­n’t be inter­preted at face value.

Let’s admit it. One taster’s 84 is anoth­er’s 86. A judge might assign higher scores ear­lier in the tast­ing than later in the morn­ing. Some might lean toward cer­tain sen­sory char­ac­ter­is­tics than oth­ers do.

They are renowned experts who form a panel rep­re­sent­ing the ulti­mate means to deter­mine extra vir­gin olive oil qual­ity. But they are human.

A few years ago, I decided to dis­con­tinue the Best in Class Award at the NYIOOC – which was bestowed on entries with the high­est score in their respec­tive cat­e­gory– due to my con­vic­tion that the scores are not suf­fi­ciently pre­cise to make that call. One excel­lent olive oil should­n’t be declared ​“bet­ter” than another excel­lent olive oil.

(Some enter­pris­ing folks nev­er­the­less felt a need to build a rank­ing based on how many awards from var­i­ous com­pe­ti­tions a brand gar­nered in a given year as if that mea­sured any­thing but the com­pa­nies’ pro­mo­tional bud­gets. But the web­sites for those rank­ings get almost no traf­fic accord­ing to met­rics tools, reveal­ing that the pub­lic does­n’t care more than I do.)

In our con­test, scores are used only to deter­mine if an oil earns an award or not – and whether it’s a Silver or Gold, and I’m not crazy about that either.

One entry could get a score of 79.8 to earn a Silver, while another gets 80.1 to win Gold. If it is tasted again a few min­utes later, the results might edge the other way around.

Olive oil com­pe­ti­tions are imper­fect, but they are the most effec­tive way to rec­og­nize pro­duc­ers for their hero­ics and edu­cate the pub­lic on qual­ity and value (though plenty of olive oil com­pe­ti­tions do nei­ther of those things).

The hid­den scores I alone can see don’t mean much on their own, but pat­terns might emerge when we zoom out to nine years of data we’ve col­lected in what amounts to the most com­pre­hen­sive sen­sory analy­sis of the world’s olive oils.

Like it or love it.

Among the award-win­ning oils, which mono­va­ri­etals made tasters swoon?

When ana­lyz­ing an entry, NYIOOC judges use the soft­ware we devel­oped that prompts them to attribute sub-scores to indi­vid­ual com­po­nents that make up the over­all result.

Things like gus­ta­tory and olfac­tory sen­sa­tions, bal­ance and har­mony, fruiti­ness, bit­ter­ness and pun­gency con­tribute to the final num­ber.

When a score reaches the 90s, it’s fair to say a beau­ti­ful oil blew the panel away. Scores in the low 70s barely made the cut­off.

Here are the aver­age scores of award-win­ning oils for the most com­mon cul­ti­vars over the last nine years. I also included the aver­age score for award-win­ning blends:

Of course, we have much more data for Picual, Arbequina and Koroneiki than for Casaliva or Tonda Iblea. Here are only the cul­ti­vars I felt had enough results to pro­duce a mean­ing­ful aver­age. More than 200 vari­eties with smaller sam­plings were omit­ted.

A cor­rectly crafted oil will con­sis­tently achieve bet­ter results than one with prob­lem­atic sen­sory char­ac­ter­is­tics, no mat­ter the vari­ety.

It also needs to be said that some cul­ti­vars are more lim­ited to regions with less vari­abil­ity in fac­tors like ter­roir, pro­duc­tion meth­ods, and qual­ity stan­dards than, for exam­ple, Arbequina, which grows world­wide.

Maybe the graph reveals spe­cific cul­ti­vars that, with the right con­di­tions and skilled pro­duc­ers, more con­sis­tently show signs of great­ness.

Or, per­haps the data sug­gests some vari­eties with sen­sory char­ac­ter­is­tics that judges have yet to appre­ci­ate fully.