What is the population of Quebec?
For the past week I have been rather puzzled by Canadian Federal Election Opinion Polls, and in particular by the difference between the numbers reported by SES and The Strategic Counsel for the Bloc Quebecois. While Strategic Counsel has reported daily numbers showing Bloc support at 13-14% nationally, SES reported a drop down to 11% -- a difference which would likely have a profound affect upon the outcome of the election. (Note to non-Canadian readers: The Bloc Quebecois is a separatist party which only runs candidates within the province of Quebec; as a result of the first-past-the-post system used in Canada, they received 12.4% of the popular vote but 17.5% of the seats in the 2004 federal election.)Now, Strategic Counsel reports their 95% confidence interval as being ±2.5%, while SES reports theirs as ±3.2%, so this difference isn't entirely outside of the possibility of random sampling error; but statistically speaking the confidence interval on the BQ vote should be roughly half the size of the confidence interval given for the entire poll -- as a result of the BQ vote share being smaller -- so these numbers are suspicious enough to deserve investigation.
Looking at the detailed numbers from the two companies provides some insight. For the days December 1st through December 4th, Strategic Counsel reports national support for the Bloc of 14% and support within Quebec of 54% (presumably these are rounded to the nearest integer percent). On December 8th, SES reports national support of 11% but provincial support of 50% for the Bloc. Ignore the difference in the days when the polls were performed, and just look at those numbers for a moment: How can a 4% difference in support within a province which is slightly less than a quarter the size of Canada translate into a 3% difference in the national result?
If you believe the numbers from Strategic Counsel, and assume that the numbers published were rounded to the nearest integer percent, then Quebec constitutes between 24.7% and 27.2% of Canada. If you believe the numbers from SES, then Quebec constitutes between 20.8% and 23.3% of Canada. In contrast, if you believe Wikipedia, the correct number is 23.6%, while Statistics Canada numbers indicate a value of 23.5%.
Obviously, without examining the detailed methodology behind the reported numbers, I can't do anything beyond pointing out that something very odd is going on with these numbers. Nevertheless, I hope this will serve as a reminder both to those who publish and to those who read such polls: Pollsters are only human, and even if you trust them not to present you with lies or damn lies, they are quite capable of making mistakes.