Scottish Referendum Opinion Polls - who is really going to win?

Everyone likes a good statistic and an opinion poll can make a fantastic headline, however, there is more to interpreting the results than many journalists care to mention. 

As the Scottish Referendum looms on the 18th September 2014, the result is so close, many commentators and researchers feel it is impossible to know which way the referendum will go: the UK is literally on a knife edge.

From a research perspective, the headline percentages are not the only thing you should be aware of. These don't tell us the true story. We have to consider how close to the true result the survey actually is... there's no point releasing the results of a survey saying 80% will say Yes, if the final result  is 40%. 

So the first thing we have to do is to get a statistical level of confidence in our findings - the normal accepted rate is to be 95% confident in our results. This level helps us to make some calculations on the necessary sample size needed. Most polls take a figure of 1,000 people from the population. 

We then have to consider the margin of error. If we asked everyone eligible to vote in Scotland (approx 4m), rather than 1,000 people, what would the true result be? Our margin of error tells us how many percent either way of our survey percentage that true result could be - and for 1,000 people it is going to be +/- 3%.  

If we take Panelbase's opinion poll from 12th September 2014 that published a vote of 46% for Yes, this went to 1,000 people with a margin of error of 3%, so you can be 95% confident that the true result in the referendum would be between 43% and 49%. Now just to confuse matters, they published 3 statistics: Yes 46%, No 47% and 7% Don't Know, so some journalists may recalculate the results based on those who have decided, and we cannot equate for the number of people who may not vote at all on the day. Previous election turnouts may give you an indication but this is a once in a lifetime referendum so those statistics may not be reliable.

So if we consider the results of a poll by Panelbase published in the Daily Mail on the 15th September 2014, this excluded undecided voters and said No was ahead with 50.6% and Yes was on 49.4%. We can be 95% confident therefore that the true value is No: 47.6% to 53.6% and Yes: 46.4% to 52.4%. We can see from this that it is impossible to call: the published statistic of No: 50.6% could equally be achieved by the Yes campaign. Results in opinion polls that are close to 50% are the hardest to call because if the result was for example 60% Yes - even with a margin of error of 3%, it would make no difference to the final result!

Whatever your view on the future of the UK, it is important to know how to look at the statistics for a full picture. The results in the next few days will be revealing but at the moment, we cannot statistically make a call. If you'd like to follow the results from the various polling companies with a good explanation of their sampling methodologies, please see the BBC's excellent reporting tracker.