RWC finals – closer contests ?

The answer to this question we suggest is that the sample size is just too small to generalise.

Certainly the points margin in the last three finals (which might be considered the real pro era) has been a maximum of 9.

The data available on the editions to date is summarised below (source: World Rugby archives).

RWC Winner Runner-up Pts Diff
2011 New Zealand France 1
2007 South Africa England 9
2003 England Australia 3
1999 Australia France 23
1995 South Africa New Zealand 3
1991 Australia England 6
1987 New Zealand France 20
Points difference
Average 9.29
Median 6
High 23
Low 1
Std Dev 8.77

Also in graphic form here RWC Finals Points Difference

RWC – did forward bulk matter ?

We often read about the importance of physical bulk in rugby. 

There is some prior academic research on this, by a group of French medical researchers, including the FFR.  The abstract of their paper is available here RWC height, mass….

We wondered about this too.  The specific questions we asked were as follows, did either:

i. average second row height, or

ii. average scrum weight

have any empirical influence on a team’s prospects of advancing through the various stages of previous RWCs, again going back to 1987 so that all RWCs were included.

What did we find ? First off, lets re-cap that what we look at is not whether say Ireland 2011 had a taller second row than Ireland 1991.  What we are interested in is between group difference, i.e. team A in 1999 vs team B in 1999, over each of the seven RWC editions held to date.

Our findings were as follows:-

i. having a taller average second row was on average a positive indicator of success in advancing from the quarter-final to the semi-final stage only;

ii. having a heavier average scrum weight was a positive indicator of success of progressing from the semi-final to the final, AND of winning the final itself.

Observation: having a taller second row or a heavier pack was a good indicator of success, if at different stages of the tournament. 

RWC – is kicking important ?

Obvious you may say, but lets go beyond opinion and have an empirical look.

To recap what we’ve researched. We looked at each stage of each RWC held to date going back to 1987.   The data was sourced from the really helpful ESPN Scrum site .

The kicking variable we looked at was the percentage of points scored with the foot, measured over the four year window (i.e. four years) prior to each RWC.

So for example, lets use 1995.  We calculated the % of points scored with the foot in all international games played by all of the teams who competed in RWC 1995.

The idea was to see to what extent these teams has been dependent on kicked points (as opposed to tries scored).

What did we find ? Well, we found that it was a good indicator of success in the latter stages of the tournament, and particularly in the final.

How might one interpret this.  It could be that as the tournament progresses, it becomes tighter (in general). As a result, teams that have a greater reliance on kicking were more likely to progress.  Put somewhat more loosely, kicking seems to have really mattered.  No surprise perhaps, but this assertion is supported by robust empirical analysis.

Observation: a high proportion of points scored from the foot (4 years prior to each RWC) has been a good indicator of success in the latter stages of past RWCs (meaning reaching the final and then becoming champion).

What is statistics and why is it relevant to rugby?


In short, (formal) statistical analysis (like we do) is relevant to rugby as it can help explain what is going on.  It allows one to go beyond pure opinion (which may be right or wrong…) and bring objectivity.

Example: “tradition is really important in rugby”… opinion

“following testing on historic RWC data, we can say with a high (statistically a 90%+ degree of confidence) that tradition (as measured empirically by the number of years since a country started playing rugby) part explains how far a country advanced in the RWCs going back to 1987.”…… that’s empirical insight, we’re trained econometricians (economic statisticians) and thats what we do.

What is Statistics ?

Many moons ago someone in ancient Greece had a great insight.

Suppose you wanted to know what was going on in a state (remember they were city states back then).  You could ask the entire population of the city state; but that would be very costly and time consuming.  One practical alternative would be to ask a sample of the population.  Clearly, this would be cheaper and faster.  The idea of taking a sample is that it can be generalised to represent the population – hence the term ‘stat’istics.

Huge care needs to be taken, however, with sample selection.  For example, one could ask 1 million 25 years olds, but that would only give a good view on what 25 year olds think.  What then what about older or younger groups ?  There would be no information about them from the sample taken. So the sample would not be representative of the population and any wider inferences drawn would be highly unlikely to be reliable.

Statisticians know that the quality of their output depends on the sample size, random selection and ideally the representative characteristics within that population.   So you will often see that e.g. political opinion polls are based on random draws from samples of x thousand people representative of the broader population….

Subject to the above, one can then perform what is known as statistical inference – that is, one can start to infer characteristics of the population based on the results of the sample (subject to confidence bands, sometimes called margins of error).

Link to Sports and Rugby

The idea was then applied to sports.  Collect data on games, run some statistical analysis, perform inference and interpret the results….

This is what we do (recognising always that there can be sampling constraints, e.g. there have only been seven editions of RWC held to date, in sampling terms, seven is a small number).  That means that caution must be taken in terms of any generalisations….

Respect to Ali W, but we prefer a different approach

This Article by former All Black Ali Williams got us thinking. The gist is that New Zealand won the RWC in 2011 because they had a core of experienced players who were battle hardened from previous RWCs.

There is no argument per se with the idea that New Zealand could improve over time. The thinking, however, raises an important question, and we think probably misses out on a key point.  The reason is that that approach compares what econometricians (aka economic statisticians or empirical economists) call within subjects, i.e. comparing the All Blacks teams of 2003, 2007, and 2011.

The key point though is that whilst those groups may have advanced, we think its much more insightful to compare ‘between‘ groups, i.e. New Zealand vs France, New Zealand vs Scotland etc, ideally at each point in time.

To illuminate, imagine a bizarre hypothetical counter-factual.  In 1995, suppose that only 1 team in the world turned professional, had access to vast resources, the best coaches, superb training facilities, and was allowed to recruit good players.  But in 1995, they started out well behind the others.  By 2015 though, if everyone else was still in the amateur era – a sort of ‘steady state’ – it wouldn’t be hard to imagine that this hypothetical team had progressed hugely to become a world leader.  They had improved, and everyone else had stood still.  So not only was their 2015 vintage better than their 1995 vintage (within progression vs themselves), but by 2015, it was better than everyone else (between progression).

This relates to the previous post on RWC design.  During each edition of RWC, teams play against each others current vintage, i.e. Scotland 2011 does not play Scotland 1999.  So the fact that Scotland 2011 may or may not have been a better team than in 1999 is not what really counts.  What really counts is how Scotland perform relative to say France or Wales if they play each other in head to head contests.  France’s 2011 vintage may also have been much better than their 1999 counterparts.

So we think it is more insightful to focus on relative differences between teams (subjects).    The fact that the All Blacks won RWC 2011 would suggest that they have been progressing at least as fast as everyone else over time, and possibly even faster.  The maintenance of that relative gap is what really matters.

How is the RWC tournament designed ?

This is a post on tournaments and tournament design.

Economists and statisticians have loved tournaments for many years. For example, many years ago two American labour (work) economists were trying to explain why a CEO’s pay jumps hugely the day they are appointed. How could someone’s marginal product jump so much, so quickly.  The solution to the puzzle came from sports. If one thinks about the race to become CEO as a promotion tournament, then the jump in remuneration as the “prize”, it all starts to make much more sense. Interestingly, this was done before sports economists had started thinking about tournament structures and incentive devices (which they subsequently have quite intensively).

Jumping on to sports tournaments. A crucial point to bear in mind is that they can be designed in many many ways, depending on the particular circumstances (e.g. purely domestic or international setting) and what the tournament organiser’s are trying to achieve (think about the World Series in Baseball as an example, or the FA Cup in football, quite different structures).

Lets take an extreme and stress purely hypothetical example to show the point. Suppose an omnipotent designer was only motivated by pure self interest. Well they could easily design a tournament such that their team only played against weak teams all the way to the final (which would be played at their home ground, with no away fans allowed in). Clearly that would push their win probability up hugely. Somewhat un-realistic yes. But it highlights that a tournament designer has many options and levers they can deploy.

RWC’s tournament format has been broadly un-changed since 1987 (yes the number of teams increased over time from 16 to 20 but the basic structure has remained the same). It can be categorised as an international, hybrid, multi-stage, single elimination tournament. Lets de-compose the technicality.

The participants represent their national rugby unions – so it is an international tournament (as an aside, thus creating the context for the research work we have done on trying to link tournament success to national fundamental characteristics such as geography and socio-economics).

It is a hybrid format in the sense that it contains two sub-formats: the opening round robin pool stage, and then the single elimination (or knock-out) phase.

It is a multi-stage tournament – in fact there are four stages: pools, quarter-final, semi-final, and final.

Single elimination is the tough part. Un-like other tournaments, there is no consolation or ‘back-door’, other than the bronze final for third and fourth place – so it is ‘single’. Elimination means that upon the completion of each stage, there is no tomorrow, teams either advance or they go home.  Its quite a tough format really.

Observation: adding the above up, from a tournament design perspective, RWC can be seen as an international hybrid multi-stage single elimination tournament.

RWC – stage effects

A quick note on the above.

One of the things we were very interested to see was the extent to which there were stage effects in past RWCs.

This simply means if different factors that we studied played a more or less influential role at different stages during the tournament historically (the tournament structure in this match list chart for RWC 2015 shows the idea. There are four main stages: pools, quarter-final, semi-final, final.)

What did we find ?  Well indeed the empirical work showed that there were stage effects. An example: we found that the team’s World Rugby ranking (see latest rankings here) was a good indicator of success at the final stage, i.e. on average the higher ranked team has become champion.  This was only so at that stage though, i.e. we could not find that it had an effect (in a statistical sense) at any other stage (e.g. pools, quarter-finals etc etc.).

What’s the insight ? Well imagine if you were trying to plot your team’s path through a tournament.  One of the things one could factor in would be that different things are likely to be influential at different stages of the tournament, depending on how far your team progresses.

Observation: there have been stage effects in past RWCs.