Monday, April 22, 2019

Alberta 2019 Election Post-mortem

Well that was fun!

How did I do?

For more than a year now I've been tracking Alberta election polls with the hope of developing a reasonably accurate prediction model. Overall, I'm happy to report that the party I predicted in the lead won in 80 out of 87 races, and my riding qualifiers broke out as follow:

  • "Solid" lead: 65/65 (100%)
  • "Likely" lead: 12/15 (80%)
  • "Lean" lead: 2/5 (40%)
  • "Toss-up" edge: 1/2 (50%)
I think this is a decent proof of concept, small "lean" sample size notwithstanding, and I want to talk a bit about what went right and what went wrong, and how I can improve if I want to keep doing this sort of thing.

First of all, the polls leading up to election day didn't turn out to be too accurate. Take a look at the province and regional splits:







Edmonton was remarkably accurate, Calgary was close, but the rest of the province and the top line results were off significantly. This is possibly a cause for concern, as it could suggest that my model was taking inaccurate data as inputs but then claiming credit for an accurate output, which it wasn't designed to do.

The NDP ended up under-performing relative to their polling numbers, and likely the only reason this didn't mess up too many election prediction models is because they under performed mostly in areas like rural Alberta, where they were predicted to lose anyway. If the polls had been that wrong about the NDP in Edmonton, say, the predictions could have been far worse.

Similarly, my model and others like me likely wouldn't have fared too well if the NDP had overperformed their polling rather than underperformed. The same amount of polling error as actually occurred, applied the other direction, could have had the NDP win the popular vote across the province.

My takeaway from this is that I need to adjust my topline polling tracker. Right now it runs under the implicit assumption that errors in individual polls will cancel each other out. This seemed reasonable given that polls are produced by different companies with different methods. That led to my full Alberta tracker having a low confidence interval for the NDP in particular, though, as several polls in a row provided the same result. If I instead make the assumption that at least part of the polling error is correlated between polls, perhaps due to something beyond their control, then the final result from election night would have still been a surprise, but far less of one. Certainly something I'll take into account next time.

Other Metrics


Overall on a riding-by-riding level, I had an error of 6.4% vote share. That's not superb, but also not far from what my testing beforehand suggested, and was factored into my uncertainty. Comparing my final projection to actual results on election night doesn't look too bad:


If we ignore the Alberta Party and the Liberals, this leads to an overall R-squared value of 0.79, which I consider respectable. It's handy to ignore the low parties because they don't have much of a spread, and will skew the coefficient of determination calculation.

Very fortunately for me, if I input the final actual regional results as though they were a poll result, my model does improve. This is a good hint that my model is behaving decently, especially so since this hasn't been the case with all other forecasters.


With the correct Calgary, Edmonton, and Rural results input as large polls, my model improved to 83/87 seats correctly predicted and an R-squared for party support per seat of 0.91. Very encouraging - too bad the polls weren't more correct!

Finally, I also provided an expected odds of winning each seat for each party. It's one thing to count a prediction as a success if you give it 100% odds of winning and it comes true, but how does one properly score oneself in the case of Calgary-Mountain View, where I gave the Liberals (10.8%), UCP (16.2%) and NDP (73%) different odds of winning, and only one (NDP) did?

In this case I've scored each riding using a Brier score. A score of 0 means a perfect prediction (100% to the winner and 0% predicted for all losers), a score of 1.0 means a perfectly wrong prediction (100% to one of the losers), and because of the math, a score of 0.19 for a complete four-way coin toss (I only predicted the four parties represented in the debate).

Overall, I scored a 0.027, which is considerably better than just guessing. It's hard to get an intuitive sense of what that score really means, but it's mathematically the same as assigning an 83.5% chance of something happening and having it come true. Not a bad prediction, but there's room to be sharpened.

How did I stack up?

So like I said, there were a lot of us predicting the election this time around. I've tried to find as many as I can, and I apologize profoundly if I've missed anyone. I've only included forecasts that had either a vote breakdown per seat or anticipated odds of winning each seat for comparison purposes.

I've reported on three main measures (seat accuracy, R-squared per seat, and prediction Brier score), and I'll present as many of those for each forecaster as I was able to determine. Different forecasters win at different categories, so it's not necessarily a clear picture as to which one of us is the "best", so I'll mostly leave room here for interpretation:



I'm not claiming to be the second best, but it's important to note that being best in one measure doesn't necessarily mean best overall. There are also harder-to-evaluate measures in play here - for instance VisualizedPolitics and TooClosetoCall allow you to input poll values to see reactions for yourself, and both improved when given more accurate data (VisualizedPolitics also got to 83 seats accurately predicted, though still with a low R-squared value).

338Canada probably rightly can claim to have been the strongest this time around, but I given the polling errors we were faced with I think it'll take several more elections to determine if anyone is really getting a significant edge consistently. This isn't the first time we've compared ourselves to each other, and I think it's an important exercise in evaluating our own models and whether there's a need for more.

9 comments:

Boante Ray said...

I just want to say a big thanks to ( CHIEF DR LUCKY ) for what he has done for me and my family by helping me to win lottery. I won Five Hundred and Thirty Three Million Us Dollars. (($533,000,000.00)). His lottery spell is the best and so amazing. My life is now balance and i am Now Very Rich As i have always wanted and i am out of doubt. Are you looking for a real and genuine spell caster to help you win big in any kind of lottery you play? you can contact Chief Dr Lucky , today all your plan to ever win Big in Lottery will Come True and work out well for you. His spell very real and genuine i can not still believe it And His spell also work very fast. Thank you so much chief Dr Lucky for dedicating your time to cast the Lottery spell for me. I got my bills paid and my debt cleared and am still mega rich. If you need a real And Genuine spell aster to help you Win lottery, kindly contact him for fast and urgent Help via email : chiefdrlucky@gmail.com or Whatsapp or Call Via +2348132777335
Website : http://chiefdrluckysolutionhome.website2.me/

Sharon Wayne said...

I want to thank Dr Emu a very powerful spell caster who help me to bring my husband back to me, few month ago i have a serious problem with my husband, to the extend that he left the house, and he started dating another woman and he stayed with the woman, i tried all i can to bring him back, but all my effort was useless until the day my friend came to my house and i told her every thing that had happened between me and my husband, then she told me of a powerful spell caster who help her when she was in the same problem I then contact Dr Emu and told him every thing and he told me not to worry my self again that my husband will come back to me after he has cast a spell on him, i thought it was a joke, after he had finish casting the spell, he told me that he had just finish casting the spell, to my greatest surprise within 48 hours, my husband really came back begging me to forgive him, if you need his help you can contact him with via email: Emutemple@gmail.com or add him up on his whatsapp +2347012841542 is willing to help any body that need his help.
Website: http://emutemple.website2.me/

Dominick said...

Well, your blog is the best one and it is very informative there is no doubt about that. But if you want to use any time tracking software then I must say group calendar is the best one for that. But you don't need to think about the price because this software is free for use and that's why we should thankful to the orapex.

Sharon Wayne said...

I want to thank Dr Emu a very powerful spell caster who help me to bring my husband back to me, few month ago i have a serious problem with my husband, to the extend that he left the house, and he started dating another woman and he stayed with the woman, i tried all i can to bring him back, but all my effort was useless until the day my friend came to my house and i told her every thing that had happened between me and my husband, then she told me of a powerful spell caster who help her when she was in the same problem I then contact Dr Emu and told him every thing and he told me not to worry my self again that my husband will come back to me after he has cast a spell on him, i thought it was a joke, after he had finish casting the spell, he told me that he had just finish casting the spell, to my greatest surprise within 48 hours, my husband really came back begging me to forgive him, if you need his help you can contact him with via email: Emutemple@gmail.com or add him up on his whatsapp +2347012841542 is willing to help any body that need his help.

Barbra Robert said...

Thanks dr jumba you promised and you did it god almighty will continue to bless you as you help people reunite their broken relationship and broken marriages .  I never thought I will smile again, My husband left me with two kids for one year, All effort to bring him back failed I thought I'm not going to see him again not until I met a lady called rose who told me about a spell caster called DR jumba, She gave me his email address and mobile number and I contacted him and he assured me that within 48hours my husband will come back to me, In less than 48hours my husband came back started begging for forgiveness saying it is the devils work, so I'm still surprise till now about this miracle,i couldn't conceive but as soon as the spell was cast,i became pregnant and gave birth to my third child,if you need any assistance from him you can contact him via: email:  wiccalovespelltools@gmail.com  or Call/WhatsApp him:  +19085174108  BLOG : blog : https://wiccaspelltools.blogspot.com/  
WEBSITE:  drjumbaspellhome.wordpress.com 

best canadian online casinos said...

Thank you for the article!

liora russell brandy said...

I completely trusted Dr Jumba from the time I spoke with him during the period my husband Left me after 1year and 3months of our marriage, Dr Jumba helped me Cast the spell work on my Relationship and gave me so much assurance and guaranteed me that he was going to bring my husband back to Me in just 15hours of the spell casting. I was so confident in his work because I have read A Lot of Testimonies online & Also for the Fact he Helped a Friend of mine Some weeks Back to get her Man Back. My faith was so strong About everything and just as Dr Jumba said to me in the beginning, my husband came back to me Asking for my Forgiveness And Also wants us to have A Happy Home together, YES MY HUSBAND IS BACK with all his hearts, Love, care, emotions and flowers and things are better now. Thankyou So much Papa, you cleared my Doubts about magic or Spells and You Also Brought Life and Happiness to me .. I would have no hesitation to recommend this powerful spell caster to anybody who is in need of help.. You can contact him through his email address: wiccalovespelltools@gmail.com OR Call ️  / WhatsApp him on: +19085174108  blog : https://wiccaspelltools.blogspot.com/  WEBSITE:  drjumbaspellhome.wordpress.com

kate said...

What could i have done if not for Priest ALUYA who restore my marriage and bring peace to my home, i will forever be grateful to you Priest ALUYA, My husband left for anther woman, he left me with our three children, i was confused and heart broken i almost lost one of my son in the processes i discus with a friend of mine and she told me about Priest ALUYA, and how he helped her in the past, so i contact him for help, and he promised me that after two day that my husband is going to call me and Apologize me for forgiveness i really didn’t believe, he told me what to do and i did every thing he asked me to do, after two day my phone rang and to my greatest surprise it was my husband he started begging me to forgive him for every thing he made me pass through, so i forgave him and now we are living happily again i say all thanks to Priest ALUYA, here is his contacts if you are passing through any marital stress i will advice you to contact him for solution, and i promise you all your problem will be solved.
(draluyasolutions@gmail.com or whatsapp...https://wa.me/message/WZDQDI75V3ZZP1

nelson said...

Do You Need A Powerful Africa Native Doctor? I Mean A Spiritual Herbalist
With Great Spirit Connected To Ancestors. Call or WhatsApp +2349058774809 Dr Obodubu Monday Is A High Voodoo Priest Master. He is a Spell Caster, Native
Doctor, Spiritual Herbalist, Powerful Sickle cell Healer, Spiritual
Astrologer, Psychic Reader, Multi Guru, Witch Doctor, Priest Of Africa
Money Ritual.
Have You Been To Native Doctor to Native Doctor Without Any Solution To
Your Problems Or Rather Add To it? Have You Been To Place To Place In View
Of A Powerful Native Doctor That Can Solve All Your Predicament, You Must
Understand That There Is A Native Doctor That Supersede All Native Doctor
Among All Native Doctor, If You Are In Search Of A Powerful Native Doctor
With Great Spiritual Powers, Then You Must Be In Search Of A Native Doctor
That Derives Power From The Marine World. That Is The Spirituality Of The
Greatest Africa Native Doctor Obodubu.
Dr Obodubu Is Recognized All Over The World Of Marine Kingdom, As One
Of The Top Fortunate And Most Powerful Native Doctor Of Charms Casts From
The Beginning Of His Ancestors ship Until Now Dr who lives Strong
Among All Other Native Doctors, There Have Never Been Any Form Of
Impossibility Beyond The Control Of Dr Obodubu. It Doesn't Matter The
Distance Of The Person With The Problems Or Situation, All You Have To Do
Is Believe
In The Native Obodubu charms Casts That Works, He Always
Warns Never To Get His Charms Casts If You Do Not Believe Or Unable To
Follow His Instruction.
It Is The Assignment Of The Native Doctor Obodubu To Offer
Services To Those In Need Of Spiritual Assistance Not Minding The Gravity
Of Your Situations Or Distance As Long As Water, Sea, Ocean, Lake, River
Sand etc. Are Near You, Then Your Problems Of Life Would Be Controlled
Under Your Foot.
If You Need Any Spiritual Help On Any of These:
Get Your Lover Back
Fruit Of The Womb
Fibroid
Business Boom
Financial Breakthrough
Get Rich Without Ritual WhatsApp +2349058774809
Do As I Say
Bad Dream
Promise And Fail
Epilepsy
Spiritual Attack
Land/Court Case
Mental Disorder
Political Appointment
Visa Approval
Cancer
Examination Success
Spend And Get Back
Good Luck
Natural Health
Hypertension
Diabesity (Diabetic+Obesity
Stroke
Sickle Cell
Impotency
Win Court Case
Promotion At Work
Commanding Tone
Protection Ring
Marriage Success
Love Ring
Favour Ring
Recover Lost Glory
Spiritual Power For Men Of God
Travel Success Ring
Job Success
Lotto/Pool Win And Many More.
Make Haste To Call or WhatsApp Dr Obodubu Monday +2349058774809. Or Mail Him to
babadoctorobodubuspiritualhome@gmail.com For Spiritual Problem Today And
You Will Surely
Get Solution To All Your Predicament.
Email: babadoctorobodubuspiritualhome@gmail.com