tag:blogger.com,1999:blog-30556045017323062582024-03-19T03:22:48.387-06:00Extreme EnginerdingQualitatively QuantifiableMichael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.comBlogger120125tag:blogger.com,1999:blog-3055604501732306258.post-13753958853899525522023-07-23T17:54:00.002-06:002023-07-23T19:29:19.639-06:00Measuring Inequality<p>It seems like with increasing frequency we hear about rising inequality, both with wealth and income distribution across our country and the world as a whole. We see articles regularly like <a href="https://www.tvo.org/article/the-rich-and-everybody-else-financial-inequality-in-canada-keeps-growing">this</a> and <a href="https://www.brookings.edu/articles/rising-inequality-a-major-issue-of-our-time/">this</a>, accurately describing it as a major issue for our times. <br /><br />With so much interest in the topic, it’s probably unsurprising that it’s a well-studied field. Before you can properly wrap your head around something you have to measure it, and in order to get policy makers to pay attention you pretty much have to boil that measurement down to a single number. So it isn’t shocking at all that economic inequality can be measured by a single value, known as the <a href="https://en.wikipedia.org/wiki/Gini_coefficient">Gini Coefficient</a>.<br /></p><h3 style="text-align: left;">I Dream of Gini</h3><p>To start looking at measuring inequality we can survey a population, rank people based on their wealth, and compare the percentage of <i>people</i> poorer than a given person to the percentage of <i>wealth held by people</i> poorer than that person. Effectively, these two values will be identical in a purely even distribution but further apart as the inequality starts to grow. If everyone has the same wealth, then the poorest 20% of the population will have 20% of the money, and the poorest 70% will have 70% of the money. If we plot this, we’d get a straight line of equality:<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR6IXdX0Ljf4Gi9Y2MfwwJ-DJeeS2obazL9iRMzeKU_mWwN0UwRhRO-DhJYhxsSIwz0LPeprSeTXPzHBjbpM-niQOyappb2gBw5IpBoOXRs_XKTiQWdAptb9nA4yFQdlB399nf_48zlA9xCkAEa_UV3-3OALw4j8Q3hWIxzEd4uzGdD9JDGp7RYQCjZJQ/s605/Equality.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="379" data-original-width="605" height="251" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR6IXdX0Ljf4Gi9Y2MfwwJ-DJeeS2obazL9iRMzeKU_mWwN0UwRhRO-DhJYhxsSIwz0LPeprSeTXPzHBjbpM-niQOyappb2gBw5IpBoOXRs_XKTiQWdAptb9nA4yFQdlB399nf_48zlA9xCkAEa_UV3-3OALw4j8Q3hWIxzEd4uzGdD9JDGp7RYQCjZJQ/w400-h251/Equality.png" width="400" /></a></div><p><br />A more unequal distribution might look like this, though, where the poorest 20% only has 5% of the wealth, and the poorest 70% only has 50%:<br /><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBmbVi-yCiVtirJgdOeSOIriI7iMo0zjaxPCoqz-1r44G02hH9PBSsgoQS7IWcf27QMPxNuovdFKFsQozG2lGb3fsxDFKiE_x0YkmN8nJ9oZnVFcJoP0k-P9nnIM6RbraVRBjG4qYZ7kGn2IelmI8_lxd2_hv4xgdeWnJo9MU8vJz8o-XkZCJiulou2_4/s605/Less%20Equal.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="379" data-original-width="605" height="251" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBmbVi-yCiVtirJgdOeSOIriI7iMo0zjaxPCoqz-1r44G02hH9PBSsgoQS7IWcf27QMPxNuovdFKFsQozG2lGb3fsxDFKiE_x0YkmN8nJ9oZnVFcJoP0k-P9nnIM6RbraVRBjG4qYZ7kGn2IelmI8_lxd2_hv4xgdeWnJo9MU8vJz8o-XkZCJiulou2_4/w400-h251/Less%20Equal.png" width="400" /></a></div><p><br />The Gini coefficient compares these two curves, the equality curve and the actual curve of the population, by comparing the area under the actual curve (B) to the area between the curve and the line of equality (A). The bigger the area between the curves (Area A), the bigger the Gini coefficient, so a Gini of 0 means a perfectly equal society, and a Gini of 1 means effectively that all the wealth is concentrated in the hands of one person.</p><p><img alt="Gini Coefficient Calculator" aria-hidden="false" class="r48jcc pT0Scc iPVvYb" height="405" src="https://uploads-cdn.omnicalculator.com/images/gini_coefficient/gini%20coefficient_01.png" style="height: 405px; margin: 0px; max-width: 500px; width: 500px;" width="500" /></p><h3 style="text-align: left;">Gini in a Bottle</h3><p>The Gini coefficient isn’t a perfect way of measuring inequality, but does a pretty good job. In the absence of social programs like a universal basic income, it’s worth pointing out that there will probably always be a non-zero Gini income coefficient, and that that’s not inherently evil. For instance, people late in their careers tend to make more money than newborn infants, and we’re generally ok with that.<br /><br />The Gini coefficient also could give the same number to different distributions, if the shape of the curve is different but still results in the same relative areas. This means that overall it’s better as a relative indicator of inequality than a pure comment on the status of a society.</p><h3 style="text-align: left;">Unleash the Gini</h3><p>As a very basic example for figuring out a Gini coefficient of our very own, we can take a look at a 10 player “Sit n Go” poker tournament. Following a common model used in online tournaments, 10 players sign up and the winner gets 50% of the pot, second place gets 30%, and third place gets 20%. Everyone else gets nothing, though hopefully has lots of fun too.<br /><br />If we wanted to plot the curve we talked about before (incidentally, called a "<a href="https://en.wikipedia.org/wiki/Lorenz_curve">Lorenz Curve</a>"), we could use the information that the bottom 70% (the 7 losers) get 0% of the wealth, the bottom 80% (7 losers + third place) own 20% of the wealth, and the bottom 90% own 50%. Put that all together and we get this graph:<br /></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUf2H7sCo6OIiCxk6UeU4187UiYEOZErkn2v73d5EJ6QW9_N6AjotA5hTkFkM2QQw_ARTKKVHmxZ4Ban02FXPXjfQs0QOSLHPfj-at6jFRZ3gq4mMg6hHPi0UpFWGuRg86GaPMXod6HQrncbVxY0j0ZkwsaSqozysYvnDTkx2cX_pC3D2VLfpiwIDSl3s/s605/Poker%2010.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="379" data-original-width="605" height="251" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUf2H7sCo6OIiCxk6UeU4187UiYEOZErkn2v73d5EJ6QW9_N6AjotA5hTkFkM2QQw_ARTKKVHmxZ4Ban02FXPXjfQs0QOSLHPfj-at6jFRZ3gq4mMg6hHPi0UpFWGuRg86GaPMXod6HQrncbVxY0j0ZkwsaSqozysYvnDTkx2cX_pC3D2VLfpiwIDSl3s/w400-h251/Poker%2010.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"></td></tr></tbody></table><p>Area A, between the curves, can now be compared to the total area of A+B, and we get a Gini coefficient of <b>0.76</b>.<br /><br />Before we get to the actual point of all this, it’s worth taking a second to reflect here. Splitting a population into ten groups and having 50% of the wealth go to the group that's <i>best at poker</i> is no basis for a system of distributing wealth. That's 50% of all cash, stocks, bonds, houses, privately held land, and super yachts. Extending the analogy, even if we were to pretend that the "best" 10% is who ends up with half the wealth, where here poker ability might correspond to concepts like hard work, diligence, education, etc, that still feels raucously unfair to end up with a distribution as shown above. And that's ignoring the fact that a significant proportion of wealth is significantly<a href="https://www.nber.org/papers/w9314"> correlated to the wealth of one's parents</a>, negating a lot of the 'hard work' argument.</p><p>So here's the issue. The Gini coefficient for the distribution of wealth in a 10 player online poker tournament is 0.76. The Gini coefficient for the distribution of <a href="https://en.wikipedia.org/wiki/List_of_sovereign_states_by_wealth_inequality">wealth in Canada</a> is 0.7<b><u>3</u></b>.</p><p>Now on the one hand, admittedly there's a little bit of room between 0.73 and 0.76. 0.76 is about the same relative inequality as in Vietnam, a bit worse than a country like Egypt (0.756) and a bit better than a country like Bolivia (0.764).</p><p>On the other hand, Canada is about the same as countries like Uganda and Liberia, which may come as a surprise to some overly self-righteous Canadians. As well, the most recent statistics are from 2019, and studies show that inequality has only <a href="https://www.imf.org/external/pubs/ft/fandd/2021/06/inequality-and-covid-19-ferreira.htm">risen during the Covid-19 pandemic</a>.We very well could be worse off than if our society had been set up as though by poker tournament.</p><p>Another thing to mention is that, as I said before, the Gini coefficient doesn't really comment on the <i>shape</i> of the Lorenz curve, just the area. And obviously Canada doesn't have 70% of people with absolutely no wealth, so maybe while the numbers are similar they don't mean the same thing, maybe it isn't quite as dire as it sounds?</p><p>Unfortunately it's not that simple. A 2012 report from the <a href="https://www.broadbentinstitute.ca/en/blog/canadas-wealth-gap-growing-poor-get-more-debt-report">Broadbent Institute</a> showed this graph for Canada:</p><p><img alt="wealth concentration canada" src="http://i.huffpost.com/gen/2050066/thumbs/o-WEALTH-CONCENTRATION-CANADA-570.jpg?6" /> </p><p>Shockingly, the top 10% (at that time) ended up with almost half the wealth, not far off from the poker example. The bottom 10% didn't just have “no wealth”, they <i>owed</i> more than they <i>owned</i>. I'd argue that if you want to think about just how rich the rich are in Canada, a 10 person poker tournament is, distressingly, actually a very good analogy.<br /></p><p>As an aside, at least in terms of poker analogies, things can always get worse. The <a href="https://www.wsop.com/tournaments/payouts/">$10,000 Main Event</a> at the World Series of Poker handily posts its payout table online, and if you do a similar analysis you get something much much worse: </p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJHVaAgGCcxNNxKqTm1ERCnqHIDaV9jdBUTpv0clGgg6e-7_ZOdMqVPQ1XRG1KjnEON69RJWT6L6TWJ8p-FRW8xOm2hfXIdSpi0ZEPfmhliPV4-zcocYd1ZnXwRXXSRk7c-WT0Y3-duUT5fjpWouSJz607OWq-DikYRcambWByNDimmi5QWRpKNQFWq7U/s605/WSOP.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="379" data-original-width="605" height="250" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJHVaAgGCcxNNxKqTm1ERCnqHIDaV9jdBUTpv0clGgg6e-7_ZOdMqVPQ1XRG1KjnEON69RJWT6L6TWJ8p-FRW8xOm2hfXIdSpi0ZEPfmhliPV4-zcocYd1ZnXwRXXSRk7c-WT0Y3-duUT5fjpWouSJz607OWq-DikYRcambWByNDimmi5QWRpKNQFWq7U/w400-h250/WSOP.png" width="400" /></a></div><p></p><p>This Gini coefficient comes in at a whopping 0.94, thankfully much higher than any real country. This is what a Lorenz curve looks like when 0.5% of the population has 50% of the wealth, and is genuinely terrifying to contemplate as a future if we don't sort things out in the real world.<br /></p><p>Canada has a long way to go in terms of wealth inequality, but obviously it gets worse too. The United States (0.852) and Russia (0.879) have absurdly high wealth inequalities. But worst of all? The world as a whole, sitting at a Gini coefficient for wealth distribution of <b><u>0.885</u></b>. We have the means to measure this and the tools to do address it, and it's well past time we do something about it.<br /></p>Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com1tag:blogger.com,1999:blog-3055604501732306258.post-20468340380756525112021-06-30T08:20:00.000-06:002021-06-30T08:20:06.384-06:00Voting Patterns for Edmonton City Council's 2017-2021 Term<p>City Council, unlike other levels of government, doesn't rely on party systems for categorizing its members. That being said, there still can be, and in fact are, patterns in how members of council vote, and with the Open Data that's available on council voting records, these patterns can be examined.</p><p>There are a lot of different ways to visualize voting patterns, and I've played around with these before (see <a href="http://www.mikerobe007.ca/2015/06/city-council-analysis.html">here</a> and <a href="http://www.mikerobe007.ca/2016/07/edmonton-city-council-votes-part-2.html">here</a> - unfortunately, since most of the visuals for this blog relied on the now-dead Google Fusion Tables, there's really not much to see). I've settled on three favourite methods for the 2017-2021 Edmonton city council term - let's take a look!</p><p>First of all, as in previous years, I've disregarded all motions that were unanimous as they provide no particular differentiating information. That leaves the 2017-2021 term with 921 non-unanimous votes to examine (at time of writing).</p><p>The first pattern-finding method I like to use it to simply look at the success rates of each member of council. How often did a vote go the way they wanted it to? This can be a sign of consensus-building, or an indicator of work put in behind the scenes (perhaps at other committees), or potentially a matter of being a part of a majority bloc that tends to vote similarly:</p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj91q0DWlGzMVc5x4piV8zxWAv2MLKkEOIDcQ_5ZmXR9M8SSp6zMyorP_D5wCXMglLE6HTzkg-AKbwS4KTIjgfR-S5wS5NjQF7y-QOBShNZ1V4lu3sTEcM9aHmmGHzRGmQaMaq8wEuToAE/" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="303" data-original-width="172" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj91q0DWlGzMVc5x4piV8zxWAv2MLKkEOIDcQ_5ZmXR9M8SSp6zMyorP_D5wCXMglLE6HTzkg-AKbwS4KTIjgfR-S5wS5NjQF7y-QOBShNZ1V4lu3sTEcM9aHmmGHzRGmQaMaq8wEuToAE/" width="136" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><br /></td></tr></tbody></table><br />While a direct comparison is perhaps unwise, these number in general follow the same pattern as my similar <a href="http://www.mikerobe007.ca/2016/07/edmonton-city-council-votes-part-2.html">2016 analysis</a>. Of members of council who were present both years, Councillor Esslinger and Mayor Iveson are again the top two and Councillor Nickel is again the lowest. Councillors Walters, Knack, and Henderson are all within 5% of their 2016 results as well, with Councillor Caterina showing a slightly larger difference from before.<p></p><p>This of course is not intended to imply anything about the effectiveness of individual members of council, and is performed without a review of the motions themselves (whether they are procedural, multiple readings of the same bylaw, etc.).</p><p>Noteworthy from the last analysis was the result that Mayor Iveson had only 'lost' <a href="https://www.cbc.ca/news/canada/edmonton/mayor-don-iveson-gets-his-way-in-most-council-votes-analysis-finds-1.3671751">17 votes</a> out of 358 non-unanimous motions in the previous term. For comparison, at the time of writing, this number is now 94 votes.</p><p>A second pattern-finding visualization is how often members of council agree with each other. For the 2017-2021 term so far, that is:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6fW6caRNpHoHsCsivjhjD3YQHstoqP9WZpnYH-1es19CHKd3r1Q90G7Ngj6hVPJP1nYrZ5uYfRRN4FFLqAJ7F1ao-OQpNgFFM__4_U4ssgUFMs0Oa0Z1pEpwDVWGNX2cFuckMQbGVCTo/" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="286" data-original-width="922" height="124" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6fW6caRNpHoHsCsivjhjD3YQHstoqP9WZpnYH-1es19CHKd3r1Q90G7Ngj6hVPJP1nYrZ5uYfRRN4FFLqAJ7F1ao-OQpNgFFM__4_U4ssgUFMs0Oa0Z1pEpwDVWGNX2cFuckMQbGVCTo/w400-h124/image.png" width="400" /></a></div><br />The result from this analysis shows that a group of six members of council agree with each other more than 80% of the time across all pairings, and that a seventh member (Councillor Henderson) is just outside with a 79% minimum agreement rate (with Councillor Hamilton). With a council size of 13, seven members is a winning majority on most motions. Certainly, there is a correlation between the top six council vote winners and this group of six members of council - whether this group is ideologically similar or just more likely to compromise and build consensus is beyond the scope of this analysis though!<p></p><p>A third and final pattern-finding visualization that I quite like is adapted from the <a href="https://en.wikipedia.org/wiki/NOMINATE_(scaling_method)">NOMINATE system</a> used to scale members of the United States Congress. It is intended to represent ideological similarities and differences between members of council in a spatial manner - members closer to each other agree more, and further apart agree less frequently:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaJwz8zwYIY1Ak3KEDeVrOgA7wjodznnvef4pvZjjj3HYOdr8uYmcdHGXQyRcPRgjUYpOWAAC02y-sMdoHr0_3wOq3hldB3Cof2A9xMs_kH3FSd5Qa1EBtl2R-LLnvECuguXoOCB0iL3c/" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="530" data-original-width="506" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaJwz8zwYIY1Ak3KEDeVrOgA7wjodznnvef4pvZjjj3HYOdr8uYmcdHGXQyRcPRgjUYpOWAAC02y-sMdoHr0_3wOq3hldB3Cof2A9xMs_kH3FSd5Qa1EBtl2R-LLnvECuguXoOCB0iL3c/w382-h400/image.png" width="382" /></a></div><p><br /></p>I'd like to stress at this moment that, as it's often tough to assign traditional political ideologies to city council bylaw amendments, this graph does not necessarily represent traditional 'left vs right wing' traits, nor traditional 'authoritarian vs libertarian' traits. The results of the graph are intended to model councillors as though their decisions are made based solely on two non-correlated factors, and the model above is oriented with the most significant factor aligned along the x-axis. <p></p><p>It's totally cool if you want to stop now, but I actually really love this model system and I want to talk about it a bit more since it gained some interest when I <a href="http://www.mikerobe007.ca/2018/09/london-city-council.html">did this for London</a>. Effectively, the NOMINATE system models both councillors <i>and</i> motions along the two axes, then assigns a probability of each councillor voting one way or another based on the relative proximity to each "side" of a debate. The algorithm then iterates thousands of times, tweaking the positions of each councillor and motion in such a way to optimize the probabilities of each decision.</p><p>The net result of this is that, using only two dimensions, this use of the NOMINATE algorithm as it stands currently accurately assigns the correct vote to each councillor 93.4% of the time. While to some of you this may not seem perfect, a model that reduces the complexity of council decisions to two factors with over 90% accuracy is something I'm quite astounded by and happy with.</p><p>For instance, last week's vote to end the mask mandate effective July 1st broke down like this based on the model. Here, the orange coloring indicates 'voted no', and blue indicates 'voted yes', with clear circles for the locations of the decision-points:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5oTQGpCV884ZJfb0o5G-xCwhkT8n-q-jUP-CSIKkf1m7jyEqTkSooZDKEfGsw805w56cRa_4BPcE_6NZMEMyIJxF9IvKThdNnKV2lNSiPJqGEQk6bH-sH0KrOHBQv1I8fMuP1PHHp7YE/" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="527" data-original-width="519" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5oTQGpCV884ZJfb0o5G-xCwhkT8n-q-jUP-CSIKkf1m7jyEqTkSooZDKEfGsw805w56cRa_4BPcE_6NZMEMyIJxF9IvKThdNnKV2lNSiPJqGEQk6bH-sH0KrOHBQv1I8fMuP1PHHp7YE/w393-h400/image.png" width="393" /></a></div><br />Here, the percentages are the model's prediction at the odds of each councillor voting the way they did. The "yes" and "no" points are shown, and the dashed line indicates the mid-way point between the two positions. In this case, the model managed to accurately capture each member's vote (where accuracy here is defined by a yes vote with more than 50% probability, or a no vote with less than 50% probability). The probability doesn't necessarily reflect the difficulty a given member of council had in making their decision, and is more of the measure of accuracy of the model. <p></p><p>By looking at all votes together, the model slowly hones in on the best placement for each member of council. Not all votes are as clean cut as this one - for instance, the vote on the solar power plant at EL Smith looked like this:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZehiwwBXM_j96UxjcQbp7CQP-LLAxoP3yV8SeY1Js3sApb735_-eDq759t_kUZT0AYG2ljWd4Uiy7779oJQZSJhmf6ceCRP5FOSJMrzyX0dIOOWhjh16_nIHcPw25blDrFQc3JrUk0Bc/" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="527" data-original-width="519" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZehiwwBXM_j96UxjcQbp7CQP-LLAxoP3yV8SeY1Js3sApb735_-eDq759t_kUZT0AYG2ljWd4Uiy7779oJQZSJhmf6ceCRP5FOSJMrzyX0dIOOWhjh16_nIHcPw25blDrFQc3JrUk0Bc/w393-h400/image.png" width="393" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: left;">You can see here that the model was very close with councillor McKeen, and effectively swapped Caterina and Dziadyk. Again, as the model is probabilistic this doesn't mean it got these 'wrong', more that having these councillors and decision points in these locations is optimized over the entire term.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">It's not a perfect model, but again I'm quite pleased with how accurately it is able to capture the voting term in only two dimensions!</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">So that's it - three different ways to look at the data, showing different aspects of what can be learned from it!</div><br /><br /><p></p>Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com7tag:blogger.com,1999:blog-3055604501732306258.post-54362409740207754062021-06-28T08:06:00.000-06:002021-06-28T08:06:21.206-06:00Which Edmonton City Councillor are you?<p>I've done this before, and had so much fun with it that I'm happy to once again present: </p><p style="text-align: center;"><b>A Buzzfeed-style quiz to get you more in touch with your elected representatives!</b></p><p style="text-align: left;"><span style="font-size: xx-small;">(it's totally ok if that doesn't excite you as much as it excites me)</span></p><p style="text-align: left;">Without further ado, here is a quiz for you to play around with. All decision points in the quiz are pulled from real votes in the 2017-2021 city council term, with information and sources provided.</p><p style="text-align: left;"><br /></p>
<iframe allowtransparency="true" frameborder="0" height="1000" scrolling="no" src="https://www.surveymonkey.com/r/XJ3NVQW" style="border: none; width: 100%;"></iframe>
<p style="text-align: left;">Hopefully that was fun!</p><p style="text-align: left;">Like I said, I've done this before for Edmonton and London, and London was <a href="https://lfpress.com/news/local-news/blogger-creates-which-london-city-councillor-are-you-quiz">far more excited about it</a>. The work that goes into these is an interesting mix of politics, whimsy, and data work.</p><p style="text-align: left;">The first step is to analyze the City of Edmonton open data set for <a href="https://data.edmonton.ca/City-Administration/2017-2021-Council-And-Committee-Meetings-Voting-Re/9iqk-6gvh">Votes and Proceedings</a>. For no discernable reason, the data set this term is inconsistent and halfway changes how votes are recorded, as well as changing how councillors are named. It's not particularly tricky to deal with, but it did have to be massaged a bit to be in a consistently usable format.</p><p style="text-align: left;">For this quiz, there's not much point in looking at unanimous procedural votes, so I focused on the 921 (at time of writing) non-unanimous votes. In an ideal world, a set of yes-no choices should require four or fewer questions in order to neatly sort into 13 possible answers (assuming approximately even splitting at each decision point). However, it's much more interesting and easy to answer the quiz when the questions are relevant and engaging. </p><p style="text-align: left;">Most of the examples I chose for this quiz have news stories attached, which in my mind was a sign of that I'd found adequately interesting votes to base this on, and as a result a user on the quiz can get to a councillor with anywhere from three to five questions, which I was satisfied with. </p><p style="text-align: left;">Hopefully you are too, because at one point in the design of this quiz one of the leading optimal votes was "That City Council waive the rules on providing notice of motion as set out in section 32 of Bylaw 18155 - Council Procedures Bylaw to allow Councillor S. Hamilton to make a motion without notice regarding the aerial mosquito program." It would've made things work <i>so well</i> but, well, it's hard to really care about it.</p><p style="text-align: left;">Each of the final results in the quiz genuinely leads to member of City Council who voted in the same unique way as the answers you provided. One assumption was made, which was that while Mike Nickel did not vote on his own censure, it was assumed that he would have voted no if he was forced to. </p><p style="text-align: left;">Hope you had fun!</p>Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com18tag:blogger.com,1999:blog-3055604501732306258.post-72983933270479843552019-04-27T14:56:00.000-06:002019-05-02T08:11:01.105-06:00Which London city councilor are you?Open data can be used for a lot of things, and public meeting minutes of elected representatives are crucial in holding representatives accountable, ensuring they represent their constituents, and promoting honesty and efficiency in our government.<br />
<br />
Or they can be used to make Buzzfeed style personality quizzes. That's what I did.<br />
<br />
We've now hit a point in the City Council <a href="https://pub-london.escribemeetings.com/MeetingsContent?MeetingViewId=1&Expanded=Council">meeting minutes from this council so far</a> where all councillors have disagreed with eachother on interesting votes at least once, which allows us to strongly differentiate between them. By presenting some of these votes, we can narrow down a few key motions that separate all the councillors, and present it in a <a href="https://en.wikipedia.org/wiki/Classification_chart">Classification Chart</a>. Since that's not as fun as a quiz, though, here it is in quiz format.<br />
<br />
Share widely, and tell me who you got! (It may take a second to load)<br />
<br />
<script data-embedid="c8qikimf" data-env="https://surveys.enalyzer.com" id="enalyzer-embed" src="https://surveys.enalyzer.com/Scripts/websitecollector.js" type="text/javascript"></script>
<br />
<div id="embed-enalyzer">
</div>
Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com165tag:blogger.com,1999:blog-3055604501732306258.post-51650994631097586932019-04-22T14:32:00.001-06:002019-04-22T14:32:01.363-06:00Alberta 2019 Election Post-mortemWell that was fun!<br />
<br />
<h3>
How did I do?</h3>
For more than a year now I've been <a href="http://www.mikerobe007.ca/p/alberta-poll-tracking.html">tracking Alberta election polls</a> with the hope of developing a reasonably accurate prediction model. Overall, I'm happy to report that the party I predicted in the lead won in 80 out of 87 races, and my riding qualifiers broke out as follow:<br />
<br />
<ul>
<li>"Solid" lead: 65/65 (100%)</li>
<li>"Likely" lead: 12/15 (80%)</li>
<li>"Lean" lead: 2/5 (40%)</li>
<li>"Toss-up" edge: 1/2 (50%)</li>
</ul>
<div>
I think this is a decent proof of concept, small "lean" sample size notwithstanding, and I want to talk a bit about what went right and what went wrong, and how I can improve if I want to keep doing this sort of thing.</div>
<div>
<br /></div>
<div>
First of all, the polls leading up to election day didn't turn out to be too accurate. Take a look at the province and regional splits:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMwbRZ9ZaIJADcib59LkcdTT2thTGdTcuEVlExbLnNWUb_gG3eWqHN0YxFr4FUe3j9X3tLa6G0qVlfuig3VZFlbpFJwSNwE3y03GXqoxkQZeUoEIIocuht_Pcx-rQB1cTpOizVzEe1Dk0/s1600/FinalAllABEB.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="378" data-original-width="578" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMwbRZ9ZaIJADcib59LkcdTT2thTGdTcuEVlExbLnNWUb_gG3eWqHN0YxFr4FUe3j9X3tLa6G0qVlfuig3VZFlbpFJwSNwE3y03GXqoxkQZeUoEIIocuht_Pcx-rQB1cTpOizVzEe1Dk0/s1600/FinalAllABEB.png" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDpGoxAXHgGTEFQG4jd0tU5hhN3OuKdY5vQOktzxRC5ytWdPWPhcXVYuD_JAahuxsXzAn-O9ct8dgmRb3s94WGA1w8ybTJHu9FjESFwXPSlKkzSNwaLvGONL47irjaGosNeIoUsNb6WAM/s1600/FinalCgy.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="378" data-original-width="578" height="209" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDpGoxAXHgGTEFQG4jd0tU5hhN3OuKdY5vQOktzxRC5ytWdPWPhcXVYuD_JAahuxsXzAn-O9ct8dgmRb3s94WGA1w8ybTJHu9FjESFwXPSlKkzSNwaLvGONL47irjaGosNeIoUsNb6WAM/s320/FinalCgy.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPoxoxHK6AxU6eDreOvVZTDCE-lOthpCQxOZemx_GJz2RhFP29sbvk1o50uiPuUP-Dwmb2lh_vBF_hNkL_sebTKUA8celxMQXwzv3ifzjsE-y1jh6TOPye8mWn-NVxifsVwgyBMAyZE5k/s1600/FinalEdm.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="378" data-original-width="578" height="209" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPoxoxHK6AxU6eDreOvVZTDCE-lOthpCQxOZemx_GJz2RhFP29sbvk1o50uiPuUP-Dwmb2lh_vBF_hNkL_sebTKUA8celxMQXwzv3ifzjsE-y1jh6TOPye8mWn-NVxifsVwgyBMAyZE5k/s320/FinalEdm.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFY-YQOtj56lhRNBISiSS8mIt_xhJN1a4_iYDjR-oogHrsG_N_NzyUtVlpRmp-Srivgq2exg53ws3HE18cR_SzwTlJBELoACbT8Wx9n1WzNHeOTgSAmBcfSn2yCUrodzoh8DXQ2vzoP48/s1600/FinalRoA.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="378" data-original-width="578" height="209" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFY-YQOtj56lhRNBISiSS8mIt_xhJN1a4_iYDjR-oogHrsG_N_NzyUtVlpRmp-Srivgq2exg53ws3HE18cR_SzwTlJBELoACbT8Wx9n1WzNHeOTgSAmBcfSn2yCUrodzoh8DXQ2vzoP48/s320/FinalRoA.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<br /></div>
<br />
Edmonton was remarkably accurate, Calgary was close, but the rest of the province and the top line results were off significantly. This is possibly a cause for concern, as it could suggest that my model was taking inaccurate data as inputs but then claiming credit for an accurate output, which it wasn't designed to do.<br />
<br />
The NDP ended up under-performing relative to their polling numbers, and likely the only reason this didn't mess up too many election prediction models is because they under performed mostly in areas like rural Alberta, where they were predicted to lose anyway. If the polls had been that wrong about the NDP in Edmonton, say, the predictions could have been far worse.<br />
<br />
Similarly, my model and others like me likely wouldn't have fared too well if the NDP had <i>over</i>performed their polling rather than <i>under</i>performed. The same amount of polling error as actually occurred, applied the other direction, could have had the NDP win the popular vote across the province.<br />
<br />
My takeaway from this is that I need to adjust my topline polling tracker. Right now it runs under the implicit assumption that errors in individual polls will cancel each other out. This seemed reasonable given that polls are produced by different companies with different methods. That led to my full Alberta tracker having a low confidence interval for the NDP in particular, though, as several polls in a row provided the same result. If I instead make the assumption that at least part of the polling error is correlated between polls, perhaps due to something beyond their control, then the final result from election night would have still been a surprise, but far less of one. Certainly something I'll take into account next time.<br />
<br />
<h3>
Other Metrics</h3>
<br />
Overall on a riding-by-riding level, I had an error of 6.4% vote share. That's not superb, but also not far from what my testing beforehand suggested, and was factored into my uncertainty. Comparing my final projection to actual results on election night doesn't look too bad:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOgD_cxmWscS-CRmGitNDorW2eldjiS-WHbkbkWHUrAIHxjbNodARvS9PAR-jv7-4lBjx52cEdDcaC-Gu7R0qFzQdmXTdTkoy-2wM1r276iHkvoYWeNpALDc6kvlSNTnFelNRIIUH_Gx4/s1600/Mefinal.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="378" data-original-width="564" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOgD_cxmWscS-CRmGitNDorW2eldjiS-WHbkbkWHUrAIHxjbNodARvS9PAR-jv7-4lBjx52cEdDcaC-Gu7R0qFzQdmXTdTkoy-2wM1r276iHkvoYWeNpALDc6kvlSNTnFelNRIIUH_Gx4/s1600/Mefinal.png" /></a></div>
<br />
If we ignore the Alberta Party and the Liberals, this leads to an overall R-squared value of 0.79, which I consider respectable. It's handy to ignore the low parties because they don't have much of a spread, and will skew the coefficient of determination calculation.<br />
<br />
Very fortunately for me, if I input the final actual regional results as though they were a poll result, my model does improve. This is a good hint that my model is behaving decently, especially so since this hasn't been the case with all other forecasters.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJdGkeFXJnBbmZscIKhxqhzA3_nLzgs0o67xjtg0ah7x-bDwYwqSd1vU3GK5esHwvXk_Kfos-iJMjE4W7g_yMwun1Q8JUHUGNVVonqGx3s5CbXY0m1m-pgiNNXOFbDl6eebpAqJxOUTJE/s1600/RealReal.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="379" data-original-width="568" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJdGkeFXJnBbmZscIKhxqhzA3_nLzgs0o67xjtg0ah7x-bDwYwqSd1vU3GK5esHwvXk_Kfos-iJMjE4W7g_yMwun1Q8JUHUGNVVonqGx3s5CbXY0m1m-pgiNNXOFbDl6eebpAqJxOUTJE/s1600/RealReal.png" /></a></div>
<br />
With the correct Calgary, Edmonton, and Rural results input as large polls, my model improved to 83/87 seats correctly predicted and an R-squared for party support per seat of 0.91. Very encouraging - too bad the polls weren't more correct!<br />
<br />
Finally, I also provided an expected odds of winning each seat for each party. It's one thing to count a prediction as a success if you give it 100% odds of winning and it comes true, but how does one properly score oneself in the case of Calgary-Mountain View, where I gave the Liberals (10.8%), UCP (16.2%) and NDP (73%) different odds of winning, and only one (NDP) did?<br />
<br />
In this case I've scored each riding using a <a href="https://en.wikipedia.org/wiki/Brier_score">Brier score</a>. A score of 0 means a perfect prediction (100% to the winner and 0% predicted for all losers), a score of 1.0 means a perfectly wrong prediction (100% to one of the losers), and because of the math, a score of 0.19 for a complete four-way coin toss (I only predicted the four parties represented in the debate).<br />
<br />
Overall, I scored a <b>0.027</b>, which is considerably better than just guessing. It's hard to get an intuitive sense of what that score really means, but it's mathematically the same as assigning an 83.5% chance of something happening and having it come true. Not a bad prediction, but there's room to be sharpened.<br />
<h3>
How did I stack up?</h3>
<div>
So like I said, there were a lot of us predicting the election this time around. I've tried to find as many as I can, and I apologize profoundly if I've missed anyone. I've only included forecasts that had either a vote breakdown per seat or anticipated odds of winning each seat for comparison purposes.</div>
<div>
<br /></div>
<div>
I've reported on three main measures (seat accuracy, R-squared per seat, and prediction Brier score), and I'll present as many of those for each forecaster as I was able to determine. Different forecasters win at different categories, so it's not necessarily a clear picture as to which one of us is the "best", so I'll mostly leave room here for interpretation:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjTNo0790ezBo1FGcN8tDhxsaNIND0ezqKXuuLLvoJQGIJ5NfK71kBYlfqO6uXGTCL7-j6UEsRI-rurnkp4DCBY5e6oIEGTd1clroVzbeSTsGlazbiFlMwNZITx2J5VkZbSq_UsFE59-c/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="144" data-original-width="435" height="105" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjTNo0790ezBo1FGcN8tDhxsaNIND0ezqKXuuLLvoJQGIJ5NfK71kBYlfqO6uXGTCL7-j6UEsRI-rurnkp4DCBY5e6oIEGTd1clroVzbeSTsGlazbiFlMwNZITx2J5VkZbSq_UsFE59-c/s320/Capture.PNG" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
I'm not claiming to be the second best, but it's important to note that being best in one measure doesn't necessarily mean best overall. There are also harder-to-evaluate measures in play here - for instance <a href="https://visualizedpolitics.com/projections/2019-alberta-election/">VisualizedPolitics</a> and <a href="https://www.tooclosetocall.ca/p/alberta-2019-simulator.html">TooClosetoCall</a> allow you to input poll values to see reactions for yourself, and both improved when given more accurate data (VisualizedPolitics also got to 83 seats accurately predicted, though still with a low R-squared value).</div>
<div>
<br /></div>
<div>
338Canada probably rightly <a href="http://blog.338canada.com/2019/04/977.html">can claim to have been the strongest</a> this time around, but I given the polling errors we were faced with I think it'll take several more elections to determine if anyone is really getting a significant edge consistently. This <a href="http://www.mikerobe007.ca/2018/06/ontario-election-wrap-up.html">isn't the first time</a> we've compared ourselves to each other, and I think it's an important exercise in evaluating our own models and whether there's a need for more.</div>
Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com30tag:blogger.com,1999:blog-3055604501732306258.post-40865694162723427672018-10-25T08:25:00.002-06:002018-10-25T09:29:24.388-06:00London Instant Runoff Breakdown London (Ontario) just had its <a href="https://lfpress.com/news/local-news/ranked-ballot-makes-history-in-london">first election using instant-runoff balloting</a>. As I've mentioned before, I'm very interested in <a href="http://www.mikerobe007.ca/2017/02/redistricting-in-alberta.html">different forms</a> of <a href="http://www.mikerobe007.ca/2015/10/electoral-reform.html">electoral reform</a>, so as a new resident of London I was intrigued as to how the vote would work out.<br />
<br />
London's system is a bit unusual inasmuch as voters can only rank their first three choices, but otherwise follows a pretty classic <a href="https://en.wikipedia.org/wiki/Instant-runoff_voting">Instant Runoff</a> system. Many of the elections resulted in first round winners, and therefore don't have a lot of room for fun analysis, but some of them went deeper and I thought it might be fun to show how the progressed in a <a href="https://en.wikipedia.org/wiki/Sankey_diagram">Sankey diagram</a>!<br />
<br />
First of all, here's <b>Ward 5</b> (my ward!):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJtON7B2zReiWYC3_Bdg7-5qfR7xDiXgcZJzY9ZhM7Mo3ba14AtJwWjPvCMO6oNQevbunu_ys2nhX_m2_pFJVGkXQ40emzezu1CPYFOcoynhys0buS16eE5or6m-hq3VIYguxOACxezp4/s1600/Ward+5+Annotated.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJtON7B2zReiWYC3_Bdg7-5qfR7xDiXgcZJzY9ZhM7Mo3ba14AtJwWjPvCMO6oNQevbunu_ys2nhX_m2_pFJVGkXQ40emzezu1CPYFOcoynhys0buS16eE5or6m-hq3VIYguxOACxezp4/s320/Ward+5+Annotated.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: left;">
As with all of the following, the leader in the first round ultimately ended up winning. Due to the lack of ability of voters to rank more than three candidates, the number of exhausted votes tends to grow quite quickly after the third round. Interesting patterns include the large number of Clarke supporters moving to Cassidy, and the relatively large number of Knott supporters preferring Warden over Cassidy at the end.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Ward 8</b></div>
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKso4VAX2LcySO-ERnfNkexn4EEBz0f4kuPs1BmzUWInOURdbGkK2sWWB9m3DFWOlHQLeSQ8kiUD8WoDYraZo-4-CIipAb8ccuBGbJ3SsDJAl7Esv0n4a0nZfiZ-6U4e4hWytkExagUzg/s1600/Ward+8+Annotated.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKso4VAX2LcySO-ERnfNkexn4EEBz0f4kuPs1BmzUWInOURdbGkK2sWWB9m3DFWOlHQLeSQ8kiUD8WoDYraZo-4-CIipAb8ccuBGbJ3SsDJAl7Esv0n4a0nZfiZ-6U4e4hWytkExagUzg/s320/Ward+8+Annotated.png" width="320" /></a></div>
<div style="text-align: left;">
This race ended closer than it began, and likely didn't see any change in leader throughout the race due to the lack of strong trends in down-ballot rankings. </div>
<div style="text-align: left;">
<br /></div>
<br />
<div class="" style="clear: both; text-align: left;">
<b>Ward 9</b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzIpbNYwO87ukM8FMfbfxc8cdUxJjyF8Cuptb0Bku0F5UfsRfWUlk5vqZNP8v0xClbsYabl6objIqZtFDDTONQvWtdWppd9zicI5VmCVEckqKmL6E0FWsdworjtTBBdV0KkE0fpaPjJnc/s1600/Ward+9+Annotated.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzIpbNYwO87ukM8FMfbfxc8cdUxJjyF8Cuptb0Bku0F5UfsRfWUlk5vqZNP8v0xClbsYabl6objIqZtFDDTONQvWtdWppd9zicI5VmCVEckqKmL6E0FWsdworjtTBBdV0KkE0fpaPjJnc/s320/Ward+9+Annotated.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
This race ended quite quickly, with Hopkins getting more than 50% of the vote by the third round after preferential support from Charlebois' supporters.</div>
<br />
<b>Ward 12</b><br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="" style="clear: both; text-align: left;">
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRccflD-cETHa1tV2upCcGZoeOl-_PNr_rbE78vzNTaVV-DwMKYgL1pajA4PHhnlRARSiU4Wb-vdB-wQZpMBq2e9l-qKt0RAsgSAU92b-VThfQ5_KuIbHC8ALLgtQQjIan7Tvw9Stl5ws/s1600/Ward+12+Annotated.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRccflD-cETHa1tV2upCcGZoeOl-_PNr_rbE78vzNTaVV-DwMKYgL1pajA4PHhnlRARSiU4Wb-vdB-wQZpMBq2e9l-qKt0RAsgSAU92b-VThfQ5_KuIbHC8ALLgtQQjIan7Tvw9Stl5ws/s320/Ward+12+Annotated.png" width="320" /></a></div>
<br />
Similar to Ward 9 - disproportionate support from Mohamed's voters to Peloza secured a win in the fourth round.</div>
<div class="separator" style="clear: both; text-align: left;">
<b>Ward 13</b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSg3wdl5XyhmM9fEBFf0dBMh-o6HY0qxMoM2mT7uSrFrQGbcl7ECBIwX79Z5ppr8mXRj-OOgEb4EWkEFzLziG7wTORSMKgLm0YNFZd3L2vlc1GRgunAI9P2ddMtPrN4GWj1Qr4_vgANMw/s1600/Ward+13+Annotated.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSg3wdl5XyhmM9fEBFf0dBMh-o6HY0qxMoM2mT7uSrFrQGbcl7ECBIwX79Z5ppr8mXRj-OOgEb4EWkEFzLziG7wTORSMKgLm0YNFZd3L2vlc1GRgunAI9P2ddMtPrN4GWj1Qr4_vgANMw/s320/Ward+13+Annotated.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
One of the tighter races of the election. Kayabaga drew large support from Warren and Hughes supporters, whereas Fyfe-Millar drew more support from Wilbee and Lundquist voters.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Ward 14</b></div>
<div class="separator" style="clear: both; text-align: left;">
<b><br /></b></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXvX-TUrLaUKpHHtEL436VFvfZ5_jEptPy4CGCOH0mbrF3A1zlyBb9NBw-IdghG4jeW0Ox9MhH0R6N4g20pFkUxXeLu_ceSB3ZDnt-H32eNXojlapMKhzs0cdqRzgUoHbkp9sldINrxCc/s1600/Ward+14+Annotated.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1600" data-original-width="1600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXvX-TUrLaUKpHHtEL436VFvfZ5_jEptPy4CGCOH0mbrF3A1zlyBb9NBw-IdghG4jeW0Ox9MhH0R6N4g20pFkUxXeLu_ceSB3ZDnt-H32eNXojlapMKhzs0cdqRzgUoHbkp9sldINrxCc/s320/Ward+14+Annotated.png" width="320" /></a></div>
<div class="" style="clear: both; text-align: left;">
Pretty straightforward - along with being the top first choice, Hillier was the preferred alternate for both Tipping and Swalwell's voters leading to a more secure finish than start.</div>
<br />
<div style="text-align: center;">
</div>
<br />
<b>Mayor</b><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlgK5MUoUr8QsF1I4Hss-Wc_9Wuq9FXtZ-Jlipz6r1fqlKT9O02sXUAVSPkfBgjA4GKYjcWzuI7rCE39jUUdTU5kmYWp5kZTELRHvFQ4Q8gazZDRHGza4Tr4c-3iLNCxKbogZFjElM1DM/s1600/Mayor+Annotated.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="800" data-original-width="1600" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlgK5MUoUr8QsF1I4Hss-Wc_9Wuq9FXtZ-Jlipz6r1fqlKT9O02sXUAVSPkfBgjA4GKYjcWzuI7rCE39jUUdTU5kmYWp5kZTELRHvFQ4Q8gazZDRHGza4Tr4c-3iLNCxKbogZFjElM1DM/s400/Mayor+Annotated.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-size: xx-small;">(Click to zoom and enhance!)</span></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
This one was far more lopsided than all the others. In the early rounds of voting, there was a small amount of jostling for positions 7-9 in the rankings, but apart from that no real changes occurred until Cheng's elimination. No abnormally strong trends in down-ticket voting occurred, though, so Holder held one throughout the end.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The city clerk has promised more detailed information to come out soon, so stay tuned for further analysis!</div>
<br />Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com29tag:blogger.com,1999:blog-3055604501732306258.post-29718522530310469062018-09-17T08:30:00.000-06:002018-09-17T08:30:41.430-06:00London City CouncilWow it's been a while since my last post. My apologies!<br />
<br />
A principal reason for this is that I've moved - I'm no longer an Edmontonian, and am now a Londoner! <a href="https://metro.co.uk/2015/10/01/11-different-towns-called-london-how-do-they-compare-to-our-capital-5416070/">London Ontario</a>, that is. This almost definitely means I won't stop posts about Edmonton, but does mean that I'll be increasing my Ontario content.<br />
<br />
London is currently in the midst of a civic election, so like any good new citizen to a city my first thought was to learn as much about the current council as I can so that I can make as informed a decision as possible. London's <a href="http://www.london.ca/city-hall/open-data/pages/open-data-data-catalogue.aspx">open data</a> is pretty good, but their <a href="https://pub-london.escribemeetings.com/MeetingsContent?MeetingViewId=1&Expanded=Council">votes and proceedings</a> aren't as organized quite as well as Edmonton's are.<br />
<br />
Nonetheless, with the votes and proceedings that are available, I thought to take a look at council relationships in London in a similar way to how I did in <a href="http://www.mikerobe007.ca/2016/07/edmonton-city-council-votes-part-2.html">Edmonton two years ago</a>.<br />
<br />
Unanimous votes aren't interesting, so I've focused this analysis on the 638 non-unanimous roll call votes as recorded in meeting minutes. First of all, let's take a look at how often each councillor agrees with each other:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQk5LA7fEB30nReOc4FC65N_CoUr0TuFnviUeB_coKinVJrdGotS8R9KEu7XrrfrWLaDCXJ6MSdeVeZMqUuusY-Q2xWcGJ6jwZa3TCBfEqi9XlZYG92IfP4Ja1rToE7Af3VGjeAwtrLek/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="279" data-original-width="756" height="147" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQk5LA7fEB30nReOc4FC65N_CoUr0TuFnviUeB_coKinVJrdGotS8R9KEu7XrrfrWLaDCXJ6MSdeVeZMqUuusY-Q2xWcGJ6jwZa3TCBfEqi9XlZYG92IfP4Ja1rToE7Af3VGjeAwtrLek/s400/Capture.PNG" width="400" /></a></div>
<br />
<br />
Matt Brown is the mayor, and currently enjoys at least 70% agreement with 11 out of 15 councillors, which isn't too shabby. In general, there appears to be a mild bloc of six people (Brown through Park) who all agree quite strongly with each other, another similar block (Park through Hubert) who do the same, and then a handful of councillors who seem to go their own way<a href="https://www.youtube.com/watch?v=6ul-cZyuYq4">.</a><br />
<br />
Another sign of consensus-building on city council is the frequency that each member of council has the outcomes of votes in line with how they voted. Again, looking only at non-unanimous votes:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMzMUJgzMphoYj18_q3fSnRDm41J7Xo4jF4a8FGKAFJxlIahJZqNvnJ1MIegRlr76Dx8lo01TrFDPOesIkO_ocP4VAupG4I0I_oc_uSnnH1oEFUpv5biUl-usdH0MstDAIxDjQORtVv7I/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="279" data-original-width="128" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMzMUJgzMphoYj18_q3fSnRDm41J7Xo4jF4a8FGKAFJxlIahJZqNvnJ1MIegRlr76Dx8lo01TrFDPOesIkO_ocP4VAupG4I0I_oc_uSnnH1oEFUpv5biUl-usdH0MstDAIxDjQORtVv7I/s1600/Capture.PNG" /></a></div>
<br />
The mayor has been on the losing side of 51 votes out of 610 in which he's been present or not recused, which suggests a reasonable level of consensus building (though not quite as high as Iveson in Edmonton).<br />
<br />
If we plot a graph of councillors, and connect them only if they agree at least 67% of the time, we get the following:<br />
<br />
<iframe frameborder="no" height="450" scrolling="no" src="https://fusiontables.google.com/embedviz?containerId=googft-gviz-canvas&viz=GVIZ&t=GRAPH&gc=false&gd=false&sdb=1&rmax=100000&q=select+col0%2C+col1%2C+col2+from+1pWzTcoVr_1OVe-3B51JfY2c9wtfgBV_OGQQ9gm5O+where+col2+%3E%3D+0.5+and+col2+%3C%3D+1&qrs=+and+col0+%3E%3D+&qre=+and+col0+%3C%3D+&qe=&uiversion=2&state=%7B%22ps%22%3A%221_5_4_-v_1_-m_b_9_-c_z_2_-k_-i_e_a_-3_a_10_-i_b_10_6_3_l_10_8_5_i_d_-1c_5_4_17_q_c_-1d_-o_7_h_-1k_6_-x_-1i_0_-1f_z_%22%2C%22cx%22%3A-20.408194416667868%2C%22cy%22%3A5.965284591952729%2C%22sw%22%3A394.05720693522375%2C%22sh%22%3A482.34324783327935%2C%22z%22%3A1.2607372667688999%7D&gco_forceIFrame=true&gco_hasLabelsColumn=true&width=500&height=500" width="450"></iframe>
<br />
The cut-off here was chosen in order to include councillor Turner while still highlighting differences in agreement rates. Unsurprisingly, councillors Turner, Helmer, and Squire are relative outsiders, with a strong cluster of the six councillors mentioned before in the center. Also, this type of graph is incredibly satisfying to play with - enjoy at your own risk!<br />
<br />
While showing relative outsiders, this plot doesn't really demonstrate any significant voting blocs. Another way to present the same data is to only connect members of council to whoever they agree with <i>the most often</i>. Doing that results in the following:<br />
<br />
<iframe frameborder="no" height="450" scrolling="no" src="https://fusiontables.google.com/embedviz?containerId=googft-gviz-canvas&viz=GVIZ&t=GRAPH&gc=false&gd=true&sdb=1&rmax=100000&q=select+col0%2C+col1%2C+col2+from+1iospb8f9i-EJjpdGm5DunNZp9oMKv1TaJiG0r-SL&qrs=+where+col0+%3E%3D+&qre=+and+col0+%3C%3D+&qe=&uiversion=2&state=%7B%22ps%22%3A%221_0_-14_-2_6_v_-3_4_-20_-11_5_1g_z_9_1v_-v_c_-1e_17_3_-2k_-1j_7_1x_1k_8_2h_-1a_e_1q_3_a_-1x_h_b_-1o_1w_2_-w_-x_d_r_-z_1_-22_-5_%22%2C%22cx%22%3A0.37961467499802637%2C%22cy%22%3A0.913173099904748%2C%22sw%22%3A585.5966956371966%2C%22sh%22%3A268.2385912519007%2C%22z%22%3A0.7469535847839434%7D&gco_forceIFrame=true&gco_hasLabelsColumn=true&width=450&height=450" width="450"></iframe>
<br />
<br />
<br />
Here we get a more interesting structure. Nearly as many people agree more often with councillor Zaifman than Mayor Brown, though there are no separated islands of voting blocs. Only two members of council agreed with each other the most mutually, Matt Brown and Maureen Cassidy, an observation that is provided without further commentary.<br />
<br />
The last way I'll look at voting patterns is to scale them using a variant of <a href="https://en.wikipedia.org/wiki/NOMINATE_(scaling_method)">NOMINATE</a>. This method was developed for analyzing US Congress voting patters, and can assign voting members to a political spectrum without needing to know what the bills being voted on were. For more information, <a href="https://voteview.com/about">this link</a> is a fascinating read.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvlZ1gUURiZimIfj-PNe24JXLAxiaU3-Ja8NHBKGMDHlE3ygXwhW1zpXH5ZYYl4bebAySzxpuFSvFhjkGZ-DLGumMOU1V-FKssb37d1R1LO6ODeqULrl9Zw3HR5_0hCcqFbDhM35euRJk/s1600/Untitled.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="379" data-original-width="573" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvlZ1gUURiZimIfj-PNe24JXLAxiaU3-Ja8NHBKGMDHlE3ygXwhW1zpXH5ZYYl4bebAySzxpuFSvFhjkGZ-DLGumMOU1V-FKssb37d1R1LO6ODeqULrl9Zw3HR5_0hCcqFbDhM35euRJk/s1600/Untitled.png" /></a></div>
<br />
Obviously a city council is going to be less partisan than a parliamentary system, but the relative placement of councillors on the graph correlates with how often the agree or disagree with each other, as well as an approximate alignment on issues. I'll detail how this was developed in a subsequent post, but the short version is that each vote is also given a numerical position, and councillors who are closer to the "yes" vote than the "no" vote are assigned probabilities to vote either way. This is then trained against the actual vote data, and thousands of iterations of machine learning later we get this distribution.<br />
<br />
Hopefully this has been an interesting glimpse into London city council. Have a fun election!Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com13London, ON, Canada42.9849233 -81.245276842.6130008 -81.8907238 43.356845799999995 -80.5998298tag:blogger.com,1999:blog-3055604501732306258.post-17541001647890613292018-06-08T09:12:00.001-06:002018-06-08T09:19:04.657-06:00Ontario Election Wrap-upThe 2018 Ontario General Election is over, and if your team won then congratulations to you!<br />
<br />
Over the last month or so I've been <a href="http://www.mikerobe007.ca/p/ontario-election-tracking.html">tracking the election polls</a> and testing out a few different ideas in order to improve a general model that I'll end up using for the <a href="http://www.mikerobe007.ca/p/alberta-poll-tracking.html">upcoming Alberta election</a>. Of course, I wasn't the only person doing this, and I was able to find at least six other sites tracking and projecting alongside.<br />
<br />
But who did the best? Can we learn anything specific about which models produce more reliable results?<br />
<br />
First of all, we can look at seat projections. As far as I could tell by mid-day June 7th, this was the seat projection distribution between the seven of us:<br />
<br />
<br />
<style type="text/css">
.tg {border-collapse:collapse;border-spacing:0;}
.tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:black;}
.tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:black;}
.tg .tg-us36{border-color:inherit;vertical-align:top}
.tg .tg-yw4l{vertical-align:top}
</style>
<br />
<table class="tg">
<tbody>
<tr>
<th class="tg-yw4l"></th>
<th class="tg-us36"><a href="https://newsinteractives.cbc.ca/onvotes/poll-tracker/">CBC</a></th>
<th class="tg-us36"><a href="http://www.tooclosetocall.ca/p/ontario-2018-simulator.html">Too Close to Call</a></th>
<th class="tg-us36"><a href="http://ontario.qc125.com/">QC125</a></th>
<th class="tg-us36"><a href="http://maps.lispop.ca/ontario_projections/">Lispop</a></th>
<th class="tg-us36"><a href="http://teddyonpolitics.blogspot.com/2018/06/final-projection-ontario.html">Teddy on Politics</a></th>
<th class="tg-us36"><a href="http://www.calculatedpolitics.com/project/2018-ontario/">Calculated Politics</a></th>
<th class="tg-us36">Extreme Enginerding</th>
<th class="tg-us36">Average</th>
<th class="tg-us36">Actual</th>
</tr>
<tr>
<td class="tg-yw4l">PC</td>
<td class="tg-us36">78</td>
<td class="tg-us36">74</td>
<td class="tg-us36">70</td>
<td class="tg-us36">69</td>
<td class="tg-us36">60</td>
<td class="tg-us36">71</td>
<td class="tg-us36">70</td>
<td class="tg-us36">70.3</td>
<td class="tg-us36">76</td>
</tr>
<tr>
<td class="tg-yw4l">NDP</td>
<td class="tg-us36">45</td>
<td class="tg-us36">46</td>
<td class="tg-us36">47</td>
<td class="tg-us36">50</td>
<td class="tg-us36">55</td>
<td class="tg-us36">44</td>
<td class="tg-us36">45</td>
<td class="tg-us36">47.4</td>
<td class="tg-us36">40</td>
</tr>
<tr>
<td class="tg-yw4l">LIB</td>
<td class="tg-us36">1</td>
<td class="tg-us36">3</td>
<td class="tg-us36">6</td>
<td class="tg-us36">4</td>
<td class="tg-us36">8</td>
<td class="tg-us36">8</td>
<td class="tg-us36">9</td>
<td class="tg-us36">5.6</td>
<td class="tg-us36">7</td>
</tr>
<tr>
<td class="tg-yw4l">GRN</td>
<td class="tg-us36">0</td>
<td class="tg-us36">1</td>
<td class="tg-us36">1</td>
<td class="tg-us36">1</td>
<td class="tg-us36">1</td>
<td class="tg-us36">1</td>
<td class="tg-us36">0</td>
<td class="tg-us36">0.7</td>
<td class="tg-us36">1</td>
</tr>
<tr>
<td class="tg-yw4l">OTH</td>
<td class="tg-us36">0</td>
<td class="tg-us36">0</td>
<td class="tg-us36">0</td>
<td class="tg-us36">0</td>
<td class="tg-us36">0</td>
<td class="tg-us36">0</td>
<td class="tg-us36">0</td>
<td class="tg-us36">0</td>
<td class="tg-us36">0</td>
</tr>
</tbody></table>
<br />
<div>
<br /></div>
<div>
Ranking these by the root sum of squares difference from the actual results, we get:</div>
<div>
<br /></div>
<div>
<ol>
<li>Calculated Politics (diff: 6.48). Their method involved seat-by-seat projections, suggesting a regional breakdown that seemed to work pretty well for them!</li>
<li>Too Close to Call (diff: 7.48). They also provided seat-by-seat projections, and had regional factors involved to project those. Also, most handily, their simulator was interactive, but putting the correct values into it actually made their predictions slightly worse (still second place at 7.87 though).</li>
<li>(Tie: CBC and Me) (diff: 8.12). We ended up with the same predictions for the NDP, but CBC was way under for the Liberals and I was quite a bit under for the PCs. My model didn't involve individual seat projections and instead just approximated historical trends for seat ranges based on party vote share, so that's a win for simplicity I suppose.</li>
<li>QC125 (diff: 9.27). Another site with seat-by-seat projections. The actual seats fell well within their expected ranges, but were all off by a little bit. I'm unsure how they came up with the seat vote projections.</li>
<li>Average (diff: 9.48). In this case, the wisdom of the crowds didn't pan out. </li>
<li>Lispop (diff: 12.57). Hypothetically they used a regional swing model similar to mine, so I'm not quite sure where the difference comes from here. It looks like they anticipated a much higher NDP voter base than actually happened.</li>
<li>Teddy on Politics (diff: 21.95). It seems like Teddy paid more attention to leader favorability numbers than most of the rest of us, and that seems to have tilted the seat distribution against him. His was the only model to predict a minority government.</li>
</ol>
<div>
For most of the models, the seat projections came directly from the popular vote estimates. If we take a look at those, we get:</div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br />
<br />
<table class="tg"><tbody>
<tr><th class="tg-yw4l"></th><th class="tg-us36"><a href="https://newsinteractives.cbc.ca/onvotes/poll-tracker/">CBC</a></th><th class="tg-us36"><a href="http://www.tooclosetocall.ca/p/ontario-2018-simulator.html">Too Close to Call</a></th><th class="tg-us36"><a href="http://ontario.qc125.com/">QC125</a></th><th class="tg-us36"><a href="http://maps.lispop.ca/ontario_projections/">Lispop</a></th><th class="tg-us36"><a href="http://teddyonpolitics.blogspot.com/2018/06/final-projection-ontario.html">Teddy on Politics</a></th><th class="tg-us36"><a href="http://www.calculatedpolitics.com/project/2018-ontario/">Calculated Politics</a></th><th class="tg-us36">Extreme Enginerding</th><th class="tg-us36">Average</th><th class="tg-us36">Actual</th></tr>
<tr><td class="tg-yw4l">PC</td><td class="tg-us36">38.7</td><td class="tg-us36">37.9</td><td class="tg-us36">37.8</td><td class="tg-us36">38</td><td class="tg-us36">37.9</td><td class="tg-us36">38.4</td><td class="tg-us36">39.8</td><td class="tg-us36">38.4</td><td class="tg-us36">40.5</td></tr>
<tr><td class="tg-yw4l">NDP</td><td class="tg-us36">35.5</td><td class="tg-us36">36</td><td class="tg-us36">36.1</td><td class="tg-us36">37</td><td class="tg-us36">36.8</td><td class="tg-us36">36.1</td><td class="tg-us36">35.9</td><td class="tg-us36">35.9</td><td class="tg-us36">33.6</td></tr>
<tr><td class="tg-yw4l">LIB</td><td class="tg-us36">19.6</td><td class="tg-us36">19.8</td><td class="tg-us36">19.7</td><td class="tg-us36">19</td><td class="tg-us36">20.9</td><td class="tg-us36">19.5</td><td class="tg-us36">19.6</td><td class="tg-us36">19.7</td><td class="tg-us36">19.6</td></tr>
<tr><td class="tg-yw4l">GRN</td><td class="tg-us36">4.9</td><td class="tg-us36">4.6</td><td class="tg-us36">5</td><td class="tg-us36">?</td><td class="tg-us36">4.5</td><td class="tg-us36">4.6</td><td class="tg-us36">5.2</td><td class="tg-us36">4.8</td><td class="tg-us36">4.6</td></tr>
<tr><td class="tg-yw4l">OTH</td><td class="tg-us36">1.3</td><td class="tg-us36">1.7</td><td class="tg-us36">1.4</td><td class="tg-us36">?</td><td class="tg-us36">0</td><td class="tg-us36">1.5</td><td class="tg-us36">1.3</td><td class="tg-us36">1.4</td><td class="tg-us36">1.8</td></tr>
</tbody></table>
</div>
<div>
<br />
Ranking these again by the same criteria we get:</div>
<div>
<br /></div>
<div>
<ol>
<li>Me! (diff: 1.15) </li>
<li>CBC (diff: 2.69)</li>
<li>Average (diff: 3.21) This is a better example of the group as a whole performing better than most individual members. This also probably makes sense as these numbers would have come mostly from the same pool of publicly available polls with a small amount of interpretation for trends and recency, as opposed to a large amount of interpretation as in the case with seat projections.</li>
<li>Calculated Politics (diff: 3.29)</li>
<li>Too Close to Call (diff: 3.56)</li>
<li>QC125 (diff: 3.73)</li>
<li>Lispop (diff: ~4.3) Note that Lispop didn't list their prediction for the green party vote total, despite projecting them to win a seat.</li>
<li>Teddy on Politics (diff: 4.37)</li>
</ol>
<div>
Overall I'm really pleased with how I did, and I've learned a few tricks to use in upcoming elections. Next up will probably be Québec, hopefully with the same group of people, and we can see if this was a fluke for me or not!<br />
<br />
Finally, here's my seat model with the actual results input as though they were one final gigantic poll at the end. Using these correct values would have resulted in the model being the most accurate seat projection of them all (diff: 4.24), which is an encouraging sign that the model itself was sound!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixdAiDu2TVW_B73jXFnDlydce1tzR8lbwvUbB_kzoJWvCP1lZk8-rSMNP60nJ1iydB0xnOOinousB1kuxdy-uSiUcKLgE2QaR1MvvBNUZvvjlCL8bW5C5LInNfFT_C7OgF-SgmACu953Q/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="454" data-original-width="682" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixdAiDu2TVW_B73jXFnDlydce1tzR8lbwvUbB_kzoJWvCP1lZk8-rSMNP60nJ1iydB0xnOOinousB1kuxdy-uSiUcKLgE2QaR1MvvBNUZvvjlCL8bW5C5LInNfFT_C7OgF-SgmACu953Q/s400/1.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
See you next election!</div>
</div>
Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com10tag:blogger.com,1999:blog-3055604501732306258.post-75262189911505727872018-04-24T10:47:00.003-06:002018-04-24T10:47:57.215-06:00Alberta Electoral DistrictsA few months ago, the Alberta Electoral Boundaries Commission released <a href="http://www.elections.ab.ca/current-news/2017/10/19/final-report-of-the-201617-electoral-boundaries-commission/">its report</a> with recommendations on how to redistrict the province for the 2019 election. As I <a href="http://www.mikerobe007.ca/2017/02/redistricting-in-alberta.html">discussed before</a>, this is an important process that occurs every eight to ten years, and is necessary for keeping the provincial electoral boundaries up to date with current population distributions.<br />
<div>
<br /></div>
<div>
As a quick aside, I'd like to thank everyone who, after reading my post on redistributing using the shortest splitline algorithm, actually <a href="https://web.archive.org/web/20170818030408/http://abebc.ca:80/submissions/">wrote in to the commission</a> to tell them to do that. Thanks guys!</div>
<div>
<br /></div>
<div>
Redistricting is always a hot topic, as it can lead to accusations of tampering or <a href="http://www.mikerobe007.ca/2015/02/voting-districts-and-gerrymandering.html">gerrymandering</a> by those in power. In Alberta the process is ostensibly done by an arms-length body, and as such when the results were unveiled in October the <a href="http://www.cbc.ca/news/canada/edmonton/electoral-division-boundaries-final-report-1.4362921">complaints were pretty tame</a> from the parties not in power. The major effect of the redistricting was to merge rural ridings in such a way that three more urban ridings were created.</div>
<div>
<br /></div>
<div>
In 2015, the poll by poll results for Alberta looked like this:</div>
<div>
<br /></div>
<iframe frameborder="no" height="700" scrolling="no" src="https://fusiontables.google.com/embedviz?q=select+col0%3E%3E2+from+1EgUL-ta3R923Efgm_aiaLMkGV0MBR3KQy0q10XIk&viz=MAP&h=false&lat=54.224211112429295&lng=-112.96236357978125&t=1&z=6&l=col0%3E%3E2&y=2&tmplt=2&hml=KML" width="500"></iframe><br />
<br />
Here, each poll is shaded a darker colour if the party won by 50% of the vote or more. It's pretty fun to zoom around in it!<br />
<br />
These polls were fitted to the 2015 riding boundaries, and if we break them out then add the votes back together according to the new 2019 boundaries, we can get a sense of what the outcome for future elections might look like. The process isn't perfect, as not all polls fit precisely into each new riding, but ultimately this is how the 2015 election is likely to have looked under the 2019 redistricting:<br />
<br />
<br />
<iframe frameborder="no" height="700" scrolling="no" src="https://fusiontables.google.com/embedviz?q=select+col10%3E%3E1+from+1KtuM296ZTXUReiatpeVKrw-7FlLHOjGwfQFovti_&viz=MAP&h=false&lat=54.67519893873337&lng=-113.66505987304691&t=1&z=6&l=col10%3E%3E1&y=2&tmplt=2&hml=KML" width="500"></iframe><br />
<br />
This map is coloured the same way as the poll map above.<br />
<br />
The impact this would have had on each party is:<br />
<br />
<ul>
<li>The total seats won by the NDP wouldn't have changed at 54</li>
<li>The total seats won by the Wildrose would have decreased from 21 to 20</li>
<li>The total seats won by the PCs would have <i>increased</i> from 10* to 12</li>
<li>The Alberta Party would have stayed at 1 seat</li>
<li>The Liberals wouldn't have won any</li>
</ul>
<div>
The next election is a little over a year away, and these will be the ridings to be determined in that election. Stay tuned as I work to better develop my <a href="http://www.mikerobe007.ca/p/alberta-poll-tracking.html">seat projection model and poll tracker</a> over the next year!</div>
Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com9tag:blogger.com,1999:blog-3055604501732306258.post-11408529512557486722017-10-23T14:41:00.001-06:002017-10-23T14:41:13.648-06:00Edmonton Election 2017Another election has come and gone, and apart from a handful of new faces the biggest news is all the new stats! Let's take a look:<br />
<div>
<br /></div>
<div>
First of all, turnout was abysmal. A total of 194,826 people voted, resulting in a voter turnout of 31.5%. The best (blue) and worst (red) areas of the city in terms of voter turnout are shown here:</div>
<div>
<br /></div>
<div>
<br /></div>
<iframe frameborder="no" height="700" scrolling="no" src="https://fusiontables.google.com/embedviz?q=select+col8%3E%3E1+from+1KuEkTcuot9VKtJXG9A9wrQit3ivhlgh4ADc0YMHh&viz=MAP&h=false&lat=53.5313114582267&lng=-113.50523482275389&t=1&z=11&l=col8%3E%3E1&y=2&tmplt=2&hml=KML" width="500"></iframe><br />
<br />
The colouring of the map is a bit funky since the mean and median are rather far apart, but it gives a decent impression of what happened. In general, it looks like neighborhoods around the river valley voted more often than neighborhoods away from it, which is interesting. The massive difference between the high (66.9%) and low (9.3%) turnout is absolutely astounding to me, and might suggest fairly significant challenges with connecting with voters in certain areas (especially if they can't see the river, apparently...).<br />
<br />
Voter turnout can also be measured in a few other ways, including attrition along the ballot. For instance, of everyone who voted, 1.5% neglected to vote for a mayoral candidate, and 1.9% neglected to vote for any council candidate. 26.3% of voters picked a Catholic schools ballot vs. 66.6% Public ballots, and even then 6.5% of Catholics and 9.8% of Publics didn't end up voting for a school trustee anyway. Oddly enough, the total number of Catholic + Public voters doesn't equal the total number of voters, so I'm not entirely sure where the remaining 7.1% of voters did for school board...<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8lM3NE11ZcC0YDOWOImuYZOGcmVknVHEl9QoCPt7r-yKCT_c0d84HhGYH7_5OgII0at_h95t6KOcKpsaMC_ZldmflXbgQcwjwFDgmjUk4KqXQRPT_xwKj-KV2thA7S8HDkxmW2y45KWM/s1600/Attrition.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="378" data-original-width="568" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8lM3NE11ZcC0YDOWOImuYZOGcmVknVHEl9QoCPt7r-yKCT_c0d84HhGYH7_5OgII0at_h95t6KOcKpsaMC_ZldmflXbgQcwjwFDgmjUk4KqXQRPT_xwKj-KV2thA7S8HDkxmW2y45KWM/s1600/Attrition.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Lighter colours represent 'under votes', or people who didn't make a pick for that particular round of voting.</td></tr>
</tbody></table>
Don Iveson was re-elected mayor with a solid victory. His support levels in Edmonton aren't dissimilar from <a href="http://www.mikerobe007.ca/2013/11/ivesons-friends.html">last election</a>, and are shown here (darker colours meaning higher support).<br />
<div>
<br /></div>
<div>
<br />
<br />
<br /></div>
<iframe frameborder="no" height="700" scrolling="no" src="https://fusiontables.google.com/embedviz?q=select+col8%3E%3E1+from+1ekB-CsMEAHEv5SeAZd-RyvulFcc7bXeog19KxVPV&viz=MAP&h=false&lat=53.53335206251722&lng=-113.49888335180663&t=1&z=11&l=col8%3E%3E1&y=2&tmplt=2&hml=KML" width="500"></iframe><br />
<br />
Iveson's support in general seems very solid in the center of the city, and a bit weaker in the north and southeast than the rest of the city. All that being said, his support ranged from 59.5-85.9% so he has a strong mandate from every part of the city.<br />
<br />
Finally, similar to l<a href="http://www.mikerobe007.ca/2013/11/ivesons-friends.html">ast election</a>, I've taken a look at which councillors' support correlates most or least with the mayor's. Last year, it turned out that a <a href="http://www.mikerobe007.ca/2015/06/city-council-analysis.html">general pattern</a> emerged where the councillors whose support most often correlated with high mayoral support also generally agreed with the mayor on votes. This year, the correlations between councillors and the mayor are:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2HA7Uj9LEwtmw1rI0K-JAvWxaMylfDJ59PGShCwhN3pmuSxKopoSew2bGSE1boGzZBSf1VMMKtxdBPsx8OjNwHHKABQpueDYaY2MM2uLUYNn1DOlNJOSqly3KQn-eAqwj4BAEoykNNUw/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="266" data-original-width="155" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2HA7Uj9LEwtmw1rI0K-JAvWxaMylfDJ59PGShCwhN3pmuSxKopoSew2bGSE1boGzZBSf1VMMKtxdBPsx8OjNwHHKABQpueDYaY2MM2uLUYNn1DOlNJOSqly3KQn-eAqwj4BAEoykNNUw/s1600/Capture.PNG" /></a></div>
<br />
I'd say this supports the theory from last election - last term, McKeen, Esslinger, Knack, Walters, and Henderson all voted alongside the mayor on <a href="http://www.mikerobe007.ca/2016/07/edmonton-city-council-votes-part-2.html">more than 80% of non-unanimous votes</a>, while Banga, Caterina, and Nickel (76%, 75%, and 46%, respectively) agreed with the mayor less frequently. While the mayor has had a strong track record of gaining majority support for non-unanimous bills, it does seem as though the candidates who do better in polls where the mayor does worse to tend on average to disagree with him more often than not.<br />
<br />
That suggests that perhaps this council will be a little bit closer in voting record than the last one - the four new councillors all showed up in the middle of the pack for mayoral correlations, so likely either they are wildcards for agreement with the mayor, or as new candidates their reputation hasn't yet been tested. Only time will tell!<br />
<br />Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com29tag:blogger.com,1999:blog-3055604501732306258.post-38778218163313402192017-06-26T16:05:00.001-06:002017-09-19T12:13:25.817-06:00Edmonton City Council Gender Parity<a href="http://www.mikerobe007.ca/2016/10/edmonton-city-council-gender-equality.html">Back in October</a> I took a quick look at the success rates of female candidates getting into city council. In 2013, 22% of candidates were female, but only one out of the twelve council seats ended up being held by a woman. The aim of that post was to investigate some of the source of the gender disparity on council - namely whether the distribution of female candidates in different races was causing the issue, or whether there was an inherent bias against female candidates.<br />
<br />
Ultimately, I determined that the relative lack of successful female council winners was more likely due to distribution of candidates across races than individual bias - without accounting for incumbency, there was no evidence of anything other than relative equal chances of winning between female and male candidates (i.e the number of female winners since 2004 is more or less what you'd expect assuming all candidates are equally likely to win).<br />
<br />
That was a pretty positive sign, as it suggests that the biggest factor holding back a demographically-balanced council is the availability of under-represented candidates to run (which is totally outside of the scope of this blog to discuss), and perhaps more importantly, the avoidance of clumping of under-represented demographics into the same few races.<br />
<br />
One of the biggest issues with the 2013 election was that five wards had no women running at all, and half of all women were clustered into two ridings. This drastically reduced the expected number of women into council, regardless of the relative proportion of candidates who put their names forward.<br />
<br />
So with all that said, I've been keeping track of candidates for the 2017 civic election which are being tracked at <a href="http://daveberta.ca/edmonton-election/">Daveberta</a>. For each candidate, I've tried to ascertain their gender in order by how they refer to themselves (political candidates <i>love</i> speaking in the third person), or how they're referred to in third party posts, and if all else fails by name and presentation assumptions. If you notice any errors, please let me know.<br />
<br />
(Last updated <b>September 19, 2017</b>)<br />
<br />
Based on the current 71 candidates, 23 are female and 48 are male (female ratio of <b>32.4%</b>, up from 22% in 2013). However, based on the distribution between wards, an expected <b>3.89</b> seats will be won by female candidates, which could be considered a relatively inefficient allocation of seats based on the ratio of candidates. <b>2 </b>wards have no women running at all.<br />
<br />
Overall, it's most likely that the number of female councillors after the election will be between 2 and 6 (90% confidence).<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2s0IHL8BKBzh04gpXHWNoeQ5St9tcZicQRR-SpnpZKMZdClLk5Bs9NzI_MdyFB9Yy38X_WMakDg5PCoAlxQDuRS4RQcn1KvkpK6gDhDHHQrUvsLPfS00zkSg6k30t2gRWpReMOl2alPo/s1600/Council.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="378" data-original-width="567" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2s0IHL8BKBzh04gpXHWNoeQ5St9tcZicQRR-SpnpZKMZdClLk5Bs9NzI_MdyFB9Yy38X_WMakDg5PCoAlxQDuRS4RQcn1KvkpK6gDhDHHQrUvsLPfS00zkSg6k30t2gRWpReMOl2alPo/s1600/Council.png" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<div class="separator" style="clear: both; text-align: center;">
Edmonton Council (32% female candidates)</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAjWuL51OlpyBlqYQbHTYJYs8PaxEtAeEV9Jr2BFK3hwTEIXtyPNDFNnevKCG7Q744lpgCEcpKyL1ewsKflhGUarrkBXsg9WXqFgq_wN3M9bAVpv2qbl_D9AjPBP9XbtHNbHRbKMN1NhI/s1600/ECSB.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="378" data-original-width="567" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAjWuL51OlpyBlqYQbHTYJYs8PaxEtAeEV9Jr2BFK3hwTEIXtyPNDFNnevKCG7Q744lpgCEcpKyL1ewsKflhGUarrkBXsg9WXqFgq_wN3M9bAVpv2qbl_D9AjPBP9XbtHNbHRbKMN1NhI/s1600/ECSB.png" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
Edmonton Catholic School Board (65% female candidates)</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKX8lm2l19HWi4B0NptBTg2u581i7qlbzIDL4ICTBV914FFskGU-7DmKMw77HaXtU75ee03X2JMCuhfGwfWYuD7eW0E6ZNsR5YfC6RqOx56XswpsfWLEFpXqAoG3QNb4oPuTZS8bMOQqg/s1600/EPSB.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="378" data-original-width="567" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKX8lm2l19HWi4B0NptBTg2u581i7qlbzIDL4ICTBV914FFskGU-7DmKMw77HaXtU75ee03X2JMCuhfGwfWYuD7eW0E6ZNsR5YfC6RqOx56XswpsfWLEFpXqAoG3QNb4oPuTZS8bMOQqg/s1600/EPSB.png" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
Edmonton Public School Board (39% female candidates)</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Now that the official nomination deadline has passed, these numbers ought to be pretty official! All in all, women running for city council are still a bit poorly distributed, leading to an expected under-representation of about 0.15 seats. On the other hand, men tend to be poorly distributed in the school board races, leading to expected over-representations of 0.28 and 0.86 seats for Catholic and Public boards respectively. All in all, the candidate distributions are fairly balanced though, and this is certainly a fairer election gender-wise than 2013.Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com6tag:blogger.com,1999:blog-3055604501732306258.post-30171569147793766992017-05-09T07:55:00.000-06:002017-05-09T07:55:09.568-06:00Next Game Wins?(Subtitle: Which Game Should You Win? Part 3)<br />
<br />
Three years ago, my friend Andrew pitched in to the blog and asked <a href="http://www.mikerobe007.ca/2014/05/which-game-should-you-win.html">which game in the playoffs was most worth winning</a>. The results were a bit inconclusive, but from it he developed a database of all playoff outcomes since 1943, so a year later I looked at the dataset again and developed Markov-style chains of <a href="http://www.mikerobe007.ca/2015/06/nhl-odds-in-best-of-7-series.html">playoff odds</a> based on different positions in the playoffs.<br />
<br />
Now that it's playoff season again, people are naturally interested more than normal in hockey and I recently overheard someone comment that, though a series was currently at 2-1 for wins, the next team to win was undoubtedly going to win the series. <br />
<br />
Good thing I have this handy database of all playoff outcomes ready, because that immediately intrigued me as to how likely it actually is that, at any given point, the next team to win a game will win the series overall. This is perhaps another way of asking the same question as before - how much does
this upcoming playoff game matter to the grand scheme of things?<br />
<br />
Before looking into the historical data, though, it's worth doing the math to see what the odds would be if the human element were removed (with all games having a 50/50 chance of going either way, and all games being independent). Obviously, if a best-of-seven playoff series is tied at 3-3, then the next game winner is guaranteed to win the series, so that's an easy starting point.<br />
<br />
From there, it's not too hard to work backwards to figure out the rest of the odds. If a series is at 3-2, then there's a 50% chance that the leading team wins (which would give them the series win, and a 100% chance therefore of winning the series), and a 50% chance that we get to a 3-3 position, where the chance of the trailing team being the overall winner is again 50%. Overall, that makes the chance that the next game winner will be the series winner (50%*100%)+(50%*50%)=<u><b>75%</b></u>.<br />
<br />
If we continue this way, then we can generate this table of values. For all following graphs, the 'home team' is the team that has home town advantage for the first two games:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxXE8wwAbIpIcXDOD6ZCFXDTPsomTpiNMbXuYdfW9EIwz7eS1wiZBVANRs9OmczKfF6kNW5mMLCk-4R8SgythVi24mmVs5kdBHYkxvCrjqQUy10Jjvuaq91ruchwhH0CFN4EtEIUObs3w/s1600/5050.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxXE8wwAbIpIcXDOD6ZCFXDTPsomTpiNMbXuYdfW9EIwz7eS1wiZBVANRs9OmczKfF6kNW5mMLCk-4R8SgythVi24mmVs5kdBHYkxvCrjqQUy10Jjvuaq91ruchwhH0CFN4EtEIUObs3w/s1600/5050.PNG" /></a></div>
<br />
<br />
So what's not surprising here is that the odds that the next team to win will be the series winner are always above 50%. That makes sense, because no matter what the position is beforehand the winner is improving their overall odds of winning the series. What's more interesting is how little games tend to matter when the series is lopsided.<br />
<br />
Of course, games aren't all independent or aren't all 50/50 toss-ups. Historically, home teams win 54.5% of games, so let's see what happens if we recreate this table with that factored in. It's a bit more complicated, but essentially the same analysis as before, to get this table:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiv5ofKh0fGSrSDpsgGBAxa4KgYtSKlQWZDmvu4LPqXK76KvewoLVYyONX46daKvPOrBLifjEWT7d6Gq85CbrtlzjDK264IUdw4yCiDvGKYgZCxmuMm7exyYAtdrqM7gmB72tZ8IiVrnK8/s1600/Home.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiv5ofKh0fGSrSDpsgGBAxa4KgYtSKlQWZDmvu4LPqXK76KvewoLVYyONX46daKvPOrBLifjEWT7d6Gq85CbrtlzjDK264IUdw4yCiDvGKYgZCxmuMm7exyYAtdrqM7gmB72tZ8IiVrnK8/s1600/Home.PNG" /></a></div>
<br />
Here we start to see the effects of the playoff structure and the pattern with which it allocates home games to different teams. For instance, when the original home team is up 3-0, the upcoming game almost doesn't matter at all, but the situation isn't quite the same if the original away team is up 3-0. Similarly, both 3-2 game situations have different values. This can be perhaps more easily rationalized - if the original home team is up 3-2, then the upcoming game is going to be in their opponent's home town, which makes it more likely that that other team will win, but if they do then it's tied coming back home, so that's less of a big deal. On the other hand, if the original away team is leading 3-2, they're more likely to win this upcoming game 6, and can lock the series up right there.<br />
<br />
Of course, this is all fun and games from a theoretical point of view, but what's actually been happening in real playoff series? Here we go:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnW-APDhK8bUh0olpMsxq28uMYys35NfDqiuq_kEDZ9-Y9V670yUoM-APyklE5IShnXcUj1vWOm67_5lpHFeFfxSIvPAk5ecA5hOMKlY8-k43EorAWpk5H0TXuP2FSNmeFIxjkbmaIGcU/s1600/Historical.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnW-APDhK8bUh0olpMsxq28uMYys35NfDqiuq_kEDZ9-Y9V670yUoM-APyklE5IShnXcUj1vWOm67_5lpHFeFfxSIvPAk5ecA5hOMKlY8-k43EorAWpk5H0TXuP2FSNmeFIxjkbmaIGcU/s1600/Historical.PNG" /></a></div>
<br />
This is definitely more interesting! Here we have a clear outlier from the theoretical projections from before, where the 'least important' game is game 5 when the original away team is up 3-1. At this point, the original home team would be playing back at home, but would be down by such a significant deficit, resulting in a situation where they end up with a fairly high 'last hurrah' win rate, before ultimately losing the series 2-4.<br />
<br />
On the other hand, there's a surprisingly high predictive score for whoever wins the game after the original home team gets up 1-0, at 74% (8% higher than what you'd expect in a coin toss scenario). I imagine this indicates that the original home team is likely to win their first game, and that if the original away team can't bounce back then the series is likely sorted out by that point (at least, in harder-to-quantify matters than you'd expect).<br />
<br />
So the answer to the question 'which playoff game is the most important' remains a solid "it depends", but now you have three different ways of looking at the question. Use them wisely, and enjoy the <a href="http://www.mikerobe007.ca/p/blog-page_14.html">2017 playoffs</a>!Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com10tag:blogger.com,1999:blog-3055604501732306258.post-6007514858417555682017-02-06T11:36:00.000-07:002017-02-06T13:08:46.605-07:00How often does the best team win the championship?Imagine we have a four team single-elimination tournament. Team A is good enough that you'd expect them to win about 80% of all games, Team B ought to win about 60%, Team C should win 40%, and Team D ought to win about 20% of games (against random opponents). If we seeded a single elimination tournament with these teams, it could look something like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkKM80YE1KI6LkVuhZK8Ekl6JMBNDGi0Z2cX1KNaKL5PlneUopPYe4Ldo4f9HEAC9VFzih3QWAR3rq3N3RuYQxzbbyxhHz7i4pXwmjG9Nopv5ByQfqSpivkBMfKmi2CR7PI-n6uCjPZfI/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkKM80YE1KI6LkVuhZK8Ekl6JMBNDGi0Z2cX1KNaKL5PlneUopPYe4Ldo4f9HEAC9VFzih3QWAR3rq3N3RuYQxzbbyxhHz7i4pXwmjG9Nopv5ByQfqSpivkBMfKmi2CR7PI-n6uCjPZfI/s1600/Capture.PNG" /></a></div>
<br />
Given the information above, how likely is it that the best team in this tournament, Team A, ends up being the winner? In other words, how effective is this tournament structure and seeding system at determining the <b>best</b> team out of the four?<br />
<br />
The first tool we'll need for this is the <a href="https://en.wikipedia.org/wiki/Log5">Log5 formula</a> - given the true winning percentages of two teams, this formula tells you the odds of a given team winning. So for instance, Team A playing Team D is pretty lopsided, and Team A has a 94.18% chance of winning that game based on this formula. Similarly, Team B and Team C are much closer in relative skill, so Team B only has a 69.23% chance of winning that game.<br />
<br />
Based on all of this, we can come up with the relative chances of any of the four teams winning the tournament overall:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeqr8ZwBo8MPpgcl25WM52lvOu7lxUdrbDpbaJKVnDP5UVY7Cw95yDDjLX8BydKwEC1kj1_mZTjRxMpVPKdAd0y_uNuIsVmVoJM6wP9p3QEp320LlVlfS8lmrJY0BWmTar8Qhm_KAbV-M/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="171" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeqr8ZwBo8MPpgcl25WM52lvOu7lxUdrbDpbaJKVnDP5UVY7Cw95yDDjLX8BydKwEC1kj1_mZTjRxMpVPKdAd0y_uNuIsVmVoJM6wP9p3QEp320LlVlfS8lmrJY0BWmTar8Qhm_KAbV-M/s320/Capture.PNG" width="320" /></a></div>
<br />
So in our hypothetical situation, this single-elimination four-player seeded tournament with known team skillsets resulted in the best team, Team A, winning <u>72.26%</u> of the time, and the worst team winning 1.06% of the time. Not shabby.<br />
<br />
However, what if you didn't know the skill levels of the teams going into the tournament? How confident could you be that the eventual winner of the tournament was, in fact, the best team when they signed up? One way to determine this is by running a Monte Carlo simulation - let's sign up four teams of random skill values, run them through a tournament exactly the same way as we just did with our sample teams, and see who the winner is. Then let's do that 10,000 times, and see how often the best team wins.<br />
<br />
The results are interesting: with randomly drawn skill values for all teams (mean= 0.5, stdev=0.13 [see below<b>**</b>]), we'd expect the winner of the tournament to be the strongest team only 44.3% of the time. About 10.5% of the time, the weakest team in the tournament would end up winning the whole thing!<br />
<br />
So is a single elimination tournament a particularly good way of determining the best overall team from a pool of four teams? Probably not. What happens if we up the ante, and have a double elimination tournament? Double elimination tournaments are exactly what they sound like - any given team needs to lose twice before being eliminated. They tend to look something like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPVzehqlYCyDZf6zPtkEyhI8lnP7SobdUxH26egq-HlJvhRVmQPTcIZ8wPDbB7sj505hTdu8JvrAIe0bgkx-SqFDMTYNW4cF4ri4TIjID4ZVxuxqoMG1pOFWQBl_4HpPTm4RgXBPB_ktU/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="252" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPVzehqlYCyDZf6zPtkEyhI8lnP7SobdUxH26egq-HlJvhRVmQPTcIZ8wPDbB7sj505hTdu8JvrAIe0bgkx-SqFDMTYNW4cF4ri4TIjID4ZVxuxqoMG1pOFWQBl_4HpPTm4RgXBPB_ktU/s320/Capture.PNG" width="320" /></a></div>
<br />
This sort of format ought to improve the chances of the best team winning, as a single unlucky (and unlikely) loss won't eliminate them too early. If we run the same sort of analysis, but with a double elimination tournament, we end up with the winner being the strongest team 51.1% of the time, and the winner being the weakest team only 7.7% of the time. A reasonable improvement all in all.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6z-f1PPW5IiRrF_lBHHqo-n7M7HPMsjWu6Z6aV_lx4g8d0DFCw8uR_vqst000Ir3QasSBih_n55PdofuKxiseE7UV0jU2aB69svoZUhAkMC1zrJrc6hNrEv9q55-7z0rUpoCQO4tp6dQ/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6z-f1PPW5IiRrF_lBHHqo-n7M7HPMsjWu6Z6aV_lx4g8d0DFCw8uR_vqst000Ir3QasSBih_n55PdofuKxiseE7UV0jU2aB69svoZUhAkMC1zrJrc6hNrEv9q55-7z0rUpoCQO4tp6dQ/s1600/Capture.PNG" /></a></div>
<br />
Unfortunately, this modest increase in chances of determining the truly best team is offset by the increased length and uncertainty in the tournament. A single elimination tournament needs three games total, whereas a double elimination tournament needs either six or seven games, depending on how the sixth one goes. This is also annoying to schedule and sell tickets for, as organizers have no way of knowing if the sixth game will be the exciting final or not.<br />
<br />
Expanding a bit, we can do the same analysis with single, double, and triple (!) elimination format tournaments for tournament of as many teams as we want. Before we continue, though, I'll mention that for a 32 team single-elimination randomly-seeded tournament, the odds that the winner will have been the best team overall are 22.0%. Keep that in mind.<br />
<br />
So why talk about these tiny little tournaments? It's to get you ready for the real deal: professional sports.<br />
<br />
Professional sports leagues generally tend to fall into a regular season and playoffs, where the regular season is used to seed teams into some sort of order and filter out the best, who end up playing in a tournament bracket of one style or another. What happens if we do the same sort of analysis for each major (and some minor) sports league?<br />
<br />
<b><u>National Basketball Association</u></b><br />
<b><u><br /></u></b>
<u>Overview:</u> 30 teams play 82 games each in the regular season. Teams are sorted into conferences and divisions. The playoffs are a 16-team single elimination tournament bracket, where each round of the playoffs consists of a best-of-seven games series for elimination. Teams are seeded within conferences, and the bracket is fixed at the end of the regular season. This gives us:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3-I_QIUTck_B-Blhs7LAD8KQ-FGyD44pkXh364s1vRSLwKUMMocUcq29fsnG8hKoKLae00Vqc63B0KIdA-XF8q8osA09ymveXVS6_e1pfShCAgU-mc-i7BvsnzQWicPUuXAsNewN88gA/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3-I_QIUTck_B-Blhs7LAD8KQ-FGyD44pkXh364s1vRSLwKUMMocUcq29fsnG8hKoKLae00Vqc63B0KIdA-XF8q8osA09ymveXVS6_e1pfShCAgU-mc-i7BvsnzQWicPUuXAsNewN88gA/s1600/Capture.PNG" /></a></div>
<br />
Overall odds of the NBA Championship being won by the season's best team: <b>45.9%.</b><br />
<b><br /></b>
<b>Pros: </b>Best-of-seven series in playoffs reduces variability in results. Not seeding teams based on division standings, and only looking at conference standings, reduces the chances of weaker teams getting into the playoffs by virtue of leading weaker divisions.<br />
<br />
<b>Cons: </b>Relatively long season for regular season, large range in potential length of playoffs.<br />
<br />
<b><u>National Hockey League</u></b><br />
<b><u><br /></u></b>
<u>Overview:</u> 30 teams play 82 games each in the regular season. Teams are sorted into conferences and divisions, and are more likely to play teams inside their divisions and conferences than outside of them. The playoffs are a 16-team single elimination tournament bracket, where each round of the playoffs consists of a best-of-seven games series for elimination. Teams are seeded within divisions such that the top three teams of each division are guaranteed a spot in the playoffs, and the highest-performing two teams of the conference that remain get in as wildcards. All of this results in the following distribution:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwkk2DY2myQDt8zS_isllBck82NldvVsLA2LGIqNk2ctKGoy7IbBEx-PfCt-QBdisB_yWUs0g1dpiLbeY5-nS8Fc33z-KF7bXYAnfGL9crQLOqmeFXp6tyThIRKRmEA-9B7tJXedxGer4/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwkk2DY2myQDt8zS_isllBck82NldvVsLA2LGIqNk2ctKGoy7IbBEx-PfCt-QBdisB_yWUs0g1dpiLbeY5-nS8Fc33z-KF7bXYAnfGL9crQLOqmeFXp6tyThIRKRmEA-9B7tJXedxGer4/s1600/Capture.PNG" /></a></div>
<br />
Overall odds the Stanley Cup winner was the season's best team: <b>45.4%</b>.<br />
<br />
<b>Pros:</b> Best-of-seven elimination in playoffs reduces variance and increases the chances of better teams triumphing.<br />
<br />
<b>Cons:</b> Relatively long season for regular season, large range in potential length of playoffs.<br />
<br />
<b>Fun fact: </b>Only real difference between NHL and NBA results is the seeding into the playoffs and how wildcards are handled, and that results in hardly any change at all.<br />
<br />
<b><u>Major League Baseball</u></b><br />
<b><u><br /></u></b>
<u>Overview:</u> 30 teams play 162 games each in the regular season. Teams are sorted into leagues and divisions, are are more likely to play teams inside their divisions and leagues than outside of them. The playoffs consist of the winners of each division and the two wildcard runners-up from the conference, and is a wonky sort of 10-team single elimination tournament where the first round are two single game wild-card playoffs, followed by four best-of-five division series, then two best-of-seven league winner series. Finally, the winners of each league play each other in a best-of-seven series to determine the winner. From this, we get:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOD_hTwbyCrHFFFU-iJlWEjeolCtlYNoYPIY96hOfDjLC45SNYk4Lhy2IidRD2KZr4Er6h-haT9aMUkWiJbKwpEwOFHmn9yyEcgA89juSozf7VIi8wbvAb7vn4zN5Q-iIp1ntI6s1Bsco/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOD_hTwbyCrHFFFU-iJlWEjeolCtlYNoYPIY96hOfDjLC45SNYk4Lhy2IidRD2KZr4Er6h-haT9aMUkWiJbKwpEwOFHmn9yyEcgA89juSozf7VIi8wbvAb7vn4zN5Q-iIp1ntI6s1Bsco/s1600/Capture.PNG" /></a></div>
<br />
Overall odds the World Series winner was the season's best team: <b>45.4%.</b><br />
<b><br /></b>
<b>Pros: </b>Teams are more likely to be correctly seeded heading into playoffs due to extensive regular season. Fewer teams in playoffs makes it less likely for weaker teams to get lucky.<br />
<br />
<b>Cons:</b> Short playoff season with sudden-death games and best-of-five series increases variance.<br />
<br />
<b>Fun fact: </b>If all rounds (including wildcard) in the MLB playoffs were best-of-seven series, the odds that the winner was actually the best team increase to 46.4%.<br />
<br />
<b><u>National Football League</u></b><br />
<b><u><br /></u></b>
<u>Overview:</u> 32 teams play 16 games each in the regular season. Teams are sorted into conferences and divisions, and are more likely to play teams inside their divisions and conferences than outside of them. The playoffs are a true 12-team single-elimination tournament, where each division leader and two runner-up wildcards are seeded from each conference. This results in:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuGt8fQy81z1FsWr5jCpMd8ZJsrNPIz2t5S1uIXJI8L08X9tf8viGMnm58lcyACUXWnuCagXn18-3NTsHAUlbEXTn_ldqI_XbBVPoV-UsOJ37Z7EHCnlkT40eM2O7LxaG5tcgCRvnc9a4/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuGt8fQy81z1FsWr5jCpMd8ZJsrNPIz2t5S1uIXJI8L08X9tf8viGMnm58lcyACUXWnuCagXn18-3NTsHAUlbEXTn_ldqI_XbBVPoV-UsOJ37Z7EHCnlkT40eM2O7LxaG5tcgCRvnc9a4/s1600/Capture.PNG" /></a></div>
<br />
Overall odds the Superbowl winner was the season's best team: <b>28.2%.</b><br />
<b><br /></b>
<b>Pros: </b>Short, fixed playoff schedule is predictable.<br />
<br />
<b>Cons: </b>Single elimination sudden death games greatly increase the chances of weaker teams winning by chance and eliminating stronger teams. Relatively short season also doesn't guarantee accurate seeding of teams heading into playoffs.<br />
<br />
<b>Fun fact: </b>Before, I mentioned that the chance of the winner of a randomly-seeded 32-team single-elimination tournament being the best team was 22.0%. That means the NFL season format isn't really all that much better than just having one six-week March Madness style showdown each season.<br />
<br />
Some more minor tournaments that are still near and dear to my heart:<br />
<br />
<b><u>Canadian Football League</u></b><br />
<b><u><br /></u></b>
<u>Overview:</u> Nine teams play 18 games each in the regular season. Teams are sorted into two divisions. The playoffs are a single-elimination 5-team tournament, where the highest-ranked team in each division gets a bye to the division finals. Teams can cross-over into other divisions if the fourth-place team in one division has more points at the end of the regular season than the third-place team in the other division. This gives us:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMBfKCV8FXLLXWatjm6addc4B_yvN72o8Si05uoFjoIz2GXPM-iIX-vp6S4B7uHV_Iz2JgLzi-K-hkCax8Sn6QDEyvkuJgEnCDXotqCt9xZhww-DgcMTnYSOqBLdAUAbjT52PfWEzh3vg/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMBfKCV8FXLLXWatjm6addc4B_yvN72o8Si05uoFjoIz2GXPM-iIX-vp6S4B7uHV_Iz2JgLzi-K-hkCax8Sn6QDEyvkuJgEnCDXotqCt9xZhww-DgcMTnYSOqBLdAUAbjT52PfWEzh3vg/s1600/Capture.PNG" /></a></div>
<br />
Overall odds the Grey Cup will go to the season's best team: <b>38.0%.</b><br />
<b><br /></b>
<b>Pros:</b> Short, fixed playoff season is predictable. Still somehow have a longer regular season than the NFL. Higher ranked teams getting a bye to the division finals helps out the stronger teams.<br />
<br />
<b>Cons:</b> Single elimination playoff format, which includes more than half the league, leads to a bit of a crapshoot. It's disappointing than the NHL with over four times as many teams is better suited to finding its best team each year.<br />
<br />
<b>Fun fact: </b>Again, this isn't substantially better than just running a 9-team randomly-seeded single elimination tournament right at the beginning of the year. It's more fun, though.<br />
<br />
<b><u>Curling</u></b><br />
<b><u><br /></u></b>
<u>Overview:</u> Major curling tournaments involve 12 teams, who play each other once each in a round robin. The top four teams are seeded into a Page playoff system, where the top two teams are in quasi-double elimination system, and the remaining teams are in single elimination. This gives us:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgiebx_14vS0yZiyezB3AvN5huztG5DmmeASEFn3I06Y-XkjKUhVj-arjQBg_zZMyALFSw6fVpCAWCWLYCzGza8fpSCCuHpX56JmErltvxWP-vPTc9Zp6-6_aHV5Ftis2KrI1TcJafedeE/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgiebx_14vS0yZiyezB3AvN5huztG5DmmeASEFn3I06Y-XkjKUhVj-arjQBg_zZMyALFSw6fVpCAWCWLYCzGza8fpSCCuHpX56JmErltvxWP-vPTc9Zp6-6_aHV5Ftis2KrI1TcJafedeE/s1600/Capture.PNG" /></a></div>
<br />
Overall odds the winner was the tournament's best team: <b>37.3%.</b><br />
<br />
<b>Pros: </b>Fixed playoff system is short and predictable. Page playoff system gives a bonus to the teams who perform best after a fair and balanced round-robin.<br />
<br />
<b>Cons: </b>Single elimination format of playoffs increases variability.<br />
<br />
<b>Fun fact: </b>Curling is fun and you should try it.<br />
<br />
So there you go. Unsurprisingly, sports leagues with longer regular seasons and best-of-seven playoff series are better suited for determining the actual best teams each season, whereas leagues with shorter seasons and single elimination tournaments as less well-suited. Now you have numbers to show for it, at least!Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com5tag:blogger.com,1999:blog-3055604501732306258.post-85540038322623766062017-02-02T16:02:00.002-07:002017-02-02T16:34:10.528-07:00Redistricting in AlbertaEvery eight to ten years, the <a href="http://abebc.ca/about/mandate/">Alberta Electoral Boundary Commission</a> meets up to reconfigure the riding boundaries for upcoming elections. This is a fairly important part of our democracy, as cities are often growing, people tend to move around a lot, and keeping our boundaries updated to reflect current population distributions is a good way to ensure that everyone is equally represented in our legislature.<br />
<br />
Fortunately enough, Alberta isn't like many US states where it is elected politicians themselves who decide where these boundaries will be drawn, which can tend to lead to gerrymandering as <a href="http://www.mikerobe007.ca/2015/02/voting-districts-and-gerrymandering.html">I've discussed before</a>. So although in Alberta districts are determined by a neutral committee and appear to avoid obvious signs of manipulation for political purposes, they aren't really all that good at making sure everyone's vote counts the same no matter where they live. For instance, the <a href="http://www.qp.alberta.ca/documents/Acts/E03.pdf">Electoral Boundaries Commission Act</a> specifies that the maximum population deviation from the average per riding is <b>25%</b>. As well, provided the area is sparsely populated enough, up to 4 districts can have populations that are as much as <b>50%</b> below the average population of a riding in the province.<br />
<br />
This led to a situation where, based on the 2011 census, the largest district had a population of 51,800 people, more than twice the size of the smallest district at 23,050. Someone living in Dunvegan-Central Peace-Notley has nearly twice the voting power of the average Albertan when it comes to provincial elections, at a population a whopping <b>45%</b> below provincial average. (As a side note, this still isn't as bad as on the federal stage, where Labrador is 73% below average, and five times less than the highest populated riding in Brantford-Brant, but that's <a href="http://www.mikerobe007.ca/2015/10/electoral-reform.html">a different story</a>.)<br />
<br />
So, as an infomercial might say at this point, "There must be a better way!"<br />
<br />
The Electoral Boundaries Commission is accepting submissions now while they begin their redistricting process, and this seems as good a time as any to determine a better solution. What is a fair way to split the province up into 87 sections each with the same population?<br />
<br />
One of the coolest solutions is to use the shortest splitline algorithm. As explained by <a href="https://www.youtube.com/watch?v=kUS9uvYyn3A">CGP Grey</a>, the shortest splitline algorithm is a repetitive process that searches for the shortest line that splits an area perfectly in two by population. Each half is then split again with the shortest line that produces equal halves, until ultimately we stop when we've gotten the desired number of sections split up, which are necessarily of exactly even population.<br />
<br />
So lets try this for Alberta. The first thing we need is a population distribution of Alberta, which Statistics Canada helpfully has lying around on their website. It looks like this:<br />
<br />
<br />
<iframe frameborder="no" height="800" scrolling="no" src="https://fusiontables.google.com/embedviz?q=select+col23%3E%3E1+from+10tFhPt39aQYYEY-_UInRF0Jv1rVEY9E6jLEM5SW1&viz=MAP&h=false&lat=54.502554007610364&lng=-114.20331798303226&t=1&z=6&l=col23%3E%3E1&y=2&tmplt=2&hml=KML" width="500"></iframe>
<br />
This is Alberta broken up into 5,711 census dissemination areas based on the 2011 census.<br />
<br />
Next up, we would normally find the shortest line that crosses Alberta in such a way that exactly half of the population is above the line, and exactly half is below the line. Since Alberta has 87 districts, though, we actually want to find the shortest line that has 44/87 of the population above it, and 43/87 of the population below it. In my (slightly optimized) model, that looked like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHfVBQO-JRIOcuTb-leojj1BmHh0xa4S3bameokcQpMyZ0OsHzkPKRtyJkI5BWAbsFAJfNRk3Q6hovPD10_HuyKPWrC9eJ-BmrPHvyM7EttQMKy2C9JrlVci1ixy4X66rs3dPDw0AhGx4/s1600/Split1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHfVBQO-JRIOcuTb-leojj1BmHh0xa4S3bameokcQpMyZ0OsHzkPKRtyJkI5BWAbsFAJfNRk3Q6hovPD10_HuyKPWrC9eJ-BmrPHvyM7EttQMKy2C9JrlVci1ixy4X66rs3dPDw0AhGx4/s1600/Split1.png" /></a></div>
<br />
Then we split each half again. The top half is an even number, so we can split it in two easily, whereas the bottom has to be split into 22/44 and 21/44 segments. That gives us this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-cOPduJY7iCZzVFuGtvcSQ0uTSlYdefyCOY-ITff1BiOWjAFwmFNieR9N5dPmZKWJYHsy4NLtVcgcsStOr_ByJMk22DXH-FbABzbmyDC9zj_puMPytvvSvj1OyZcCbsneT3bz_o_qjec/s1600/Split2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-cOPduJY7iCZzVFuGtvcSQ0uTSlYdefyCOY-ITff1BiOWjAFwmFNieR9N5dPmZKWJYHsy4NLtVcgcsStOr_ByJMk22DXH-FbABzbmyDC9zj_puMPytvvSvj1OyZcCbsneT3bz_o_qjec/s1600/Split2.png" /></a></div>
<br />
And so on and so forth until we've split all of Alberta up into equal segments. The final result of this ends up being this lovely stained glass window:<br />
<br />
<br />
<iframe frameborder="no" height="800" scrolling="no" src="https://fusiontables.google.com/embedviz?q=select+col3+from+1knX8y6LvyAEo9NO8VF_57ZHuWzXmTG5qZ8XBRLDi&viz=MAP&h=false&lat=54.576751252166204&lng=-114.29497891803743&t=1&z=6&l=col3&y=2&tmplt=2&hml=KML" width="500"></iframe>
<br />
<br />
Of course, things can't always be perfect no matter how hard you try, so this is a solution for Alberta that has a maximum population in each riding of <b>0.38%</b>. The largest riding has 42,052 people in it, and the smallest has 41,752. This is a solution to split up Alberta that has a maximum voter variance that is 118 times smaller than we have now, and a coefficient of variation that is 68 times smaller.<br />
<br />
Also, just because the map was drawn with straight lines doesn't mean it has to stay like that. If we go back to our census dissemination area shapes from Stats Canada, we can convert an Edmonton distribution from this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWu1wV8NmlJrd73pKLUeSQGZgvTD-zyTeKU4DHXKMtYjAoxJSLER86RTDwEfTnXjHZBjRx_9EPMkEaA3KcMXhTNcSgjQRuPXPCekqHYzYnvcEj3Z9d05H8vvPgOute7WCohahTHWKQ7Zw/s1600/EdmontonSplitline.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWu1wV8NmlJrd73pKLUeSQGZgvTD-zyTeKU4DHXKMtYjAoxJSLER86RTDwEfTnXjHZBjRx_9EPMkEaA3KcMXhTNcSgjQRuPXPCekqHYzYnvcEj3Z9d05H8vvPgOute7WCohahTHWKQ7Zw/s320/EdmontonSplitline.PNG" width="271" /></a></div>
<br />
To this:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiv-ScTCt6u0T6XvPCTHvMZnM2MKNSJ0PUMnl5tYpkOGrZ0gmwLb5GsZtcwFTQH3EMDpPPdYzsQTdSGi1ssgNMb7IeFXGlvXch4AuBMtsbtv-ES1G1iRiZ7RMCt_UBIobFzcRRgFGUbMHY/s1600/EdmontonSplitline2.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiv-ScTCt6u0T6XvPCTHvMZnM2MKNSJ0PUMnl5tYpkOGrZ0gmwLb5GsZtcwFTQH3EMDpPPdYzsQTdSGi1ssgNMb7IeFXGlvXch4AuBMtsbtv-ES1G1iRiZ7RMCt_UBIobFzcRRgFGUbMHY/s320/EdmontonSplitline2.PNG" width="272" /></a></div>
<br />
Which is actually starting to look pretty reasonable. Neighborhoods are kept together, and the areas are looking relatively compact.<br />
<br />
The shortest splitline method is an objective and fair way to distribute votes such that everyone's votes are counted equally. I was able to redistrict all of Alberta using Excel - no fancy programming skills are needed. There's no reason that we can't have redistricting being as boring as updating census data and having a computer spit out a single solution each time we need it.<br />
<br />
That being said, there are still some objections people could have with it - for instance, it doesn't necessarily give a hoot about municipal boundaries. Take Red Deer for example: after applying the algorithm to Alberta, Red Deer got sort of unfortunately split into four districts, each of which includes substantial amounts of surrounding countryside:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiU-__mKmpdgfxWHkTp0e_QyZFpkS-RtZerP5WcYoqkSd33C0-MvUs5Ibeq8ABpfN9y_0_RDPhPzPI_wiAD07CWOo8mxsI2E0hkoiUz-tXqzm4wcYVHXDpoiOP_LemEnNbvyVl8QZAuIfM/s1600/Red+Deer+Split.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiU-__mKmpdgfxWHkTp0e_QyZFpkS-RtZerP5WcYoqkSd33C0-MvUs5Ibeq8ABpfN9y_0_RDPhPzPI_wiAD07CWOo8mxsI2E0hkoiUz-tXqzm4wcYVHXDpoiOP_LemEnNbvyVl8QZAuIfM/s320/Red+Deer+Split.PNG" width="273" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Oh no.</td></tr>
</tbody></table>
This would have people in Innisfail voting alongside southwest Red Deer, and people in Saskatchewan River Crossing voting alongside west Red Deer. That's probably a bit messed up. In cases like this, I think it's probably fair to use the splitline algorithm to get to a starting point, and then massaging the districts as needed to make sure they make a bit more sense. Because the algorithm got within a 0.38% maximum population variance to start with, it is likely quite straightforward after to swap around some areas as needed to keep the popluation variance still small. For instance, Public Interest Alberta <a href="http://www.pialberta.org/make_every_vote_equal_in_redrawing_of_electoral_boundaries">recommends a maximum population deviation of 5%</a>, which I'd suggest is reasonably easy to achieve if we're starting from a point where our deviation is essentially 0%.<br />
<br />
So if I've convinced you that using algorithms to redistrict our population can lead to fairer, objective, even distributions of our political districts, and that those are things worth having in our democracy, head on over to the <a href="http://abebc.ca/submissions/">Commission's website</a> and leave them a submission before February 8th!Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com6tag:blogger.com,1999:blog-3055604501732306258.post-35276097379569910672016-10-27T15:46:00.001-06:002016-10-27T15:46:34.925-06:00Edmonton City Council Gender EqualityIn the 2013 civic election, 79 candidates ran for mayor and city council, 17 of which were women. The election resulted in one woman getting elected out of a council of 13. Though women represent 51% of the city's population, they represented only 22% of candidates, and resulted in 8% of council seats. With results like this, it's little surprise that groups like <a href="https://www.equalvoice.ca/ab_north.cfm">Equal Voice</a> are calling for improvements to our system, including encouraging a larger diversity of candidates and promoting a more balanced and representative city government.<br />
<br />
Taking a deeper look at these results shows some interesting trends though. For instance, in 5 wards in 2013, there were no women running at all, and half of all female candidates were clustered in races in two highly contested wards. This suggests that, while 22% of all candidates were female, the distribution of female candidates may have already been predisposed to a lower number of female winners in the end. Let's take a look.<br />
<br />
There were 12 wards and one mayoral race in 2013 for city council, and the proportion of female candidates per race ranged from 0-43%. Assuming any given candidate has an equal chance of winning any given race (an assumption we'll check later), this is the expected distribution of female winners:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjizCCT9A37-RlazYG_DdT5cUqORf0tISS3Goo8SLcK0i0zmx5jF4MMmYsiBrF9ixzWLk9WiAQwxhxXRU-V4eNLcbqkeuZDOqlU8TP3Ipdjy4GoqFK1VXNxKq1gYYC7h6c-71gyycM9XLs/s1600/2013+Expected.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjizCCT9A37-RlazYG_DdT5cUqORf0tISS3Goo8SLcK0i0zmx5jF4MMmYsiBrF9ixzWLk9WiAQwxhxXRU-V4eNLcbqkeuZDOqlU8TP3Ipdjy4GoqFK1VXNxKq1gYYC7h6c-71gyycM9XLs/s1600/2013+Expected.png" /></a></div>
<br />
As previously mentioned, there was absolutely no chance of there being anywhere from 9-13 women on council, as 5 races were contested solely by men. Based on the uneven distribution of candidates in the remaining races, there was an expected 2.01 female councillors last year, or 15% of council. So while the number of women on council was still less than expected, it was closer than what we might have expected based on the total number of female candidates. Instead of being 14% lower than what we might expect from candidate distribution, we were 8% lower.<br />
<br />
So does this mean that female candidates are 8% less likely to get elected than male candidates? That's really hard to say, and it turns out we don't have enough data yet. One way we can check is by looking at the p-value of our outcome - what's the chance that we could have gotten something as bad as the result we did, assuming our null hypothesis (that women are as likely to get elected as men) is true? In this case, the p-value is 0.37. Essentially, our data set is small enough that any result between 0-4 female councillors wouldn't have been all that much of a surprise (and in fact, 6-7 would have been an indication of an opposite effect). So let's not worry about significance yet, and instead look at more elections!<br />
<br />
Edmonton's civic elections elect people to mayor, council, and public and catholic school boards. If we do the same analysis for all three councils for the last four elections, we get a chart that looks like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoAkMrmjlROdgrlUK_3JqnihV7zA9pdt6CX-YZFHymCqBPimUd6-xEfx1wWdDzI4gIJGrPZuj_KIsAHnX9ZLDH4GwGwmVMZYRXGR-9ALOqG7Pyt14_Tg3sW9xPIEZilSCi4I0bqcoKtGA/s1600/Historical+Deviation.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoAkMrmjlROdgrlUK_3JqnihV7zA9pdt6CX-YZFHymCqBPimUd6-xEfx1wWdDzI4gIJGrPZuj_KIsAHnX9ZLDH4GwGwmVMZYRXGR-9ALOqG7Pyt14_Tg3sW9xPIEZilSCi4I0bqcoKtGA/s1600/Historical+Deviation.png" /></a></div>
<br />
This suggests a lot of things, including:<br />
<br />
<ul>
<li>City Council results over the last 4 elections haven't shown more than a 10% deviation one way or another. More importantly, the p-values for each election have been totally reasonable.</li>
<li>There's a lot of variation in the Public School Board elections. This is partially explainable based on how small the Board is, so any variation will be magnified from a percentage basis. On the other hand, that level of variation isn't present in the (smaller) Catholic Board...</li>
<li>Catholic School Board elections haven't shown an anti-woman bias in this data set.</li>
</ul>
<div>
So, interestingly enough, of the 12 discrete votes that I looked at, 4 had a slight anti-woman bias, and 8 were perhaps slightly pro-women. Essentially, what this suggests is that <b>female candidates are just as likely to get elected as male candidates</b>. If we add up all the results since 2014 into one graph (of 402 candidates running for 114* positions over the last four years), we get the following distribution:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjo5k7b5H-LcEP2PGV7h2yy7C0-ifzt7vJfNp2LJ5ziNMHW_965ZB3urluxw2HB90Y4jlWYxbjnh24_wRKilrPkIxUjCJ4kNhLrqCUrgDarnyblj7LaKmv0ziYhV1N3svZdfqwX_tnvNvA/s1600/Total+Winners.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjo5k7b5H-LcEP2PGV7h2yy7C0-ifzt7vJfNp2LJ5ziNMHW_965ZB3urluxw2HB90Y4jlWYxbjnh24_wRKilrPkIxUjCJ4kNhLrqCUrgDarnyblj7LaKmv0ziYhV1N3svZdfqwX_tnvNvA/s1600/Total+Winners.png" /></a></div>
<div>
<br /></div>
<div>
Overall, 46 women have been elected to 114* electable seats, where the candidate distribution and chance would expect us to have elected 42.5. The p-value assuming an equal electability between women and men is 0.22, so no, meninists, there isn't a <i>pro</i>-woman bias either. These results are pretty much what we'd expect given the candidate distribution we've had. This general conclusion holds true across city council (p=0.29):</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDcC_sK1bb0ymgfzmMkf7nFe9qDztN97yT-c1qtkYhZmMbjdLQxPny936VOWKo0eCeZGO2OFxMY15tVkfVLfXwc2xxIDsA-tTecc_FsLmPH8A8dhdvzoC39QnuJPbyWn43b9XpzJPCGjI/s1600/Total+Council.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDcC_sK1bb0ymgfzmMkf7nFe9qDztN97yT-c1qtkYhZmMbjdLQxPny936VOWKo0eCeZGO2OFxMY15tVkfVLfXwc2xxIDsA-tTecc_FsLmPH8A8dhdvzoC39QnuJPbyWn43b9XpzJPCGjI/s1600/Total+Council.png" /></a></div>
<br />
And Public Schools (p=0.19):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-L4ubqgNR6R0N7yRRYeyle2C5N_qbsIPQH_C2lJagccC6Gp9FN8Jy50mEs-miicak4o5wvNQgJskL3YVt9n6jDUQMT5Hm3d_G8zHnwrsmnaTmlGYllXVBddPr4PncBnTE84PHaj-POJs/s1600/Total+EPSB.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-L4ubqgNR6R0N7yRRYeyle2C5N_qbsIPQH_C2lJagccC6Gp9FN8Jy50mEs-miicak4o5wvNQgJskL3YVt9n6jDUQMT5Hm3d_G8zHnwrsmnaTmlGYllXVBddPr4PncBnTE84PHaj-POJs/s1600/Total+EPSB.png" /></a></div>
<div>
<br /></div>
<div>
Though intriguingly enough breaks down at Catholic Schools* (p=0.03):</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhM3JwKBmAxBMoFG2XZh-xG8o3iFrO-iaGTlkoq9hOp__j0xjsONUvV5FHyWHp7L8hKn0_yW0AxguUiQ1s95iXANeE1fhKMEgIKT5pahHeFqBY5QU9jFA_8PUPXg2q-ubLLZFIE__q2GPQ/s1600/Total+Catholic.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhM3JwKBmAxBMoFG2XZh-xG8o3iFrO-iaGTlkoq9hOp__j0xjsONUvV5FHyWHp7L8hKn0_yW0AxguUiQ1s95iXANeE1fhKMEgIKT5pahHeFqBY5QU9jFA_8PUPXg2q-ubLLZFIE__q2GPQ/s1600/Total+Catholic.png" /></a></div>
<blockquote class="tr_bq">
*: Here it's worth mentioning that before 2010, the Catholic election system was really weird and had a wildcard winner from whoever had the most votes but wasn't elected in their ward. This was particularly silly seeing as not all wards were the same size, so I've ignored the wildcard seat and victor for the purposes of this analysis.</blockquote>
The fortunate summary of all of this is that there's no evidence that any system is rigged against female candidates. That being said, the proportion of women elected to civic office in the last years is just under a third of total offices filled, which isn't even remotely balanced. The best way to get a more representative council is to have more under-represented demographics put forward their candidacy, so if you know anyone who might be interested or qualified (of whichever underrepresented group you choose), I strongly encourage you to encourage them to run.<br />
Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com6tag:blogger.com,1999:blog-3055604501732306258.post-16783821438476631462016-08-17T10:00:00.001-06:002016-08-17T10:00:56.807-06:00Edmonton Bike Safety<a href="http://www.theglobeandmail.com/news/alberta/edmonton-mayor-laments-slow-progress-on-bike-lanes/article31211499/">Bicycles in Edmonton</a> have been in the <a href="http://edmontonjournal.com/news/local-news/bike-lane-controversy-poised-to-hit-edmontons-109-street">news quite a bit recently</a>, particularly given the success of new <a href="http://www.metronews.ca/news/edmonton/2016/07/26/edmonton-bike-lane-network-could-rival-calgary.html">bicycle development in Calgary</a>. Bicycle lanes in Edmonton have been proposed, installed, removed, <a href="http://www.cbc.ca/news/canada/edmonton/city-moves-quickly-to-remove-guerilla-bike-lane-1.3638757">illegally painted</a>, removed again, and blocked in council quite a bit in the last few years. Frustrations between cyclists, city planners, and drivers have gotten to a boiling point recently, and I think it's safe to say that whichever side of the debate you're on you're likely sick of it all. But please keep reading!<br />
<br />
With all that said, things have recently gotten a bit more interesting from a data point of view. A month ago I was made aware of a <a href="http://spacing.ca/edmonton/2015/01/27/bike-collisions-edmonton/">data set of cycling injuries and incidents from 2009-2014</a> from the nice folks at Spacing Edmonton, which were analyzed by them as well as (more recently) the group over at <a href="https://slowstreets.wordpress.com/2016/07/26/using-collision-mapping-to-indicate-cycling-desire-lines/">Slow Streets</a>.<br />
<br />
Specifically, the people at Slow Streets made the claim that injury hot spots indicate where more cyclists are travelling, showing cyclist 'desire lines' which would be prime targets for bicycle infrastructure. However, a quick look at the map suggests that the streets with supposedly lots of bicycle traffic are also the roads with lots of <i>vehicle</i> traffic. Hypothetically, even if all streets had the same bicycle traffic, we might expect a similar distribution since one might think that more cars might lead to more interactions with cars.<br />
<br />
So let's take a look and check this hypothesis. Fortunately, Edmonton has a nice map of <a href="http://www.edmonton.ca/transportation/RoadsTraffic/Flow_map_2014_AAWDT.pdf">average annual weekday traffic</a> for major roadways. I combined the map data of all 1,070 cyclist injuries from 2009-2014 with the map of all streets that had traffic volume stats, and ended up with this result:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9x7BJjVhvofW8jkG6X8uzqPMgIzc6iz8O-dciVBUHb1YXX7oUapBfwjUUbUEf7N6lb2DcwrGRrCDk0ZZufVQ0fspneh18c4PfnINEeAt9EugFIraN1i-jp0J-PsPQnzuTXn8uGAEOm4M/s1600/Bike+Injuries.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9x7BJjVhvofW8jkG6X8uzqPMgIzc6iz8O-dciVBUHb1YXX7oUapBfwjUUbUEf7N6lb2DcwrGRrCDk0ZZufVQ0fspneh18c4PfnINEeAt9EugFIraN1i-jp0J-PsPQnzuTXn8uGAEOm4M/s1600/Bike+Injuries.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Error bars represent the 95% confidence interval for injury rate.</td></tr>
</tbody></table>
<br />
It looks as though there is a decisive link between <i>vehicular</i> traffic on a road and the number of cyclist injuries. As the city doesn't seem to have any specific information on bicycle ride distributions, it's hard to say with any certainty if the Slow Streets analysis is correct. Either way, it's clear that wherever cyclists mix with lots of cars, we get lots of injuries. This analysis ended up looking at 571 km of major roads with traffic data, which were responsible for 760 of the injuries recorded from 2009-2014.<br />
<br />
But hey, that's not all! Edmonton also has a map of everyone's favorite (or least favorite) things - <a href="http://www.edmonton.ca/transportation/RoadsTraffic/CycleEdmontonMap_13092013.pdf">bike lanes</a>!<br />
<br />
From the map, Edmonton's road bike-friendliness can be broken down into four different types. There are separated shared use pathways, painted on-road bike lanes, signed on-road bike lanes, and plain old normal streets. So what does my previous analysis look like if we split road types up by their bicycle infrastructure? Why, this:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpw3ZP6-5nFfIuyv5nDH65pIZAWtdnVM17c1KXQRBjynSBuDwuBos6Y8b3ScE_44bfVfnXLvKtg-NBBDSZLDxPCAUbU4EANjK1VA0_D3kFb8V8YRV0uYtksVIBo_uKlfjDVD8huME4poc/s1600/Bike+Injuries+Lanes.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpw3ZP6-5nFfIuyv5nDH65pIZAWtdnVM17c1KXQRBjynSBuDwuBos6Y8b3ScE_44bfVfnXLvKtg-NBBDSZLDxPCAUbU4EANjK1VA0_D3kFb8V8YRV0uYtksVIBo_uKlfjDVD8huME4poc/s1600/Bike+Injuries+Lanes.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Again, error bars are the 95% confidence interval. Basically, ignore the green bars...</td></tr>
</tbody></table>
<br />
What are some takeaways from this? Well, first of all, major roads very infrequently have <i>signed</i> on-road bike lanes, so there's far too much variability for a proper analysis of them (green on the graph). Far more common are roads with separated shared bike paths (red), or no infrastructure at all (grey). From this, we get the firm (and hopefully not unreasonable at all) conclusion that biking on separated, wide, shared pathways for bikes is safer than biking on a normal road with traffic, by a factor of about 2.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgit4S9gwVOrcKRhdTr2vU9LTasqqn1iIsjNhq48jZHAn4S8O-d5SxLvLROrVyv8chgPVGnKXMvr1pwviYbHwEezHJKdWLStDHL2iBD5FjbCKG-adO5EFciYQRPyHdIfRKid-ZqqssPVjo/s1600/Bike+Injuries+Legend.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgit4S9gwVOrcKRhdTr2vU9LTasqqn1iIsjNhq48jZHAn4S8O-d5SxLvLROrVyv8chgPVGnKXMvr1pwviYbHwEezHJKdWLStDHL2iBD5FjbCKG-adO5EFciYQRPyHdIfRKid-ZqqssPVjo/s320/Bike+Injuries+Legend.png" width="271" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">From the City of Edmonton <a href="http://www.edmonton.ca/transportation/RoadsTraffic/CycleEdmontonMap_13092013.pdf">bike map</a></td></tr>
</tbody></table>
<br />
However, an interesting conclusion from this is that it's extremely hard to make the argument that painted bike lanes are safer than normal roads. In fact, in some cases, it looks quite a bit safer to bike on non-bike-laned roads. Weird.<br />
<br />
What might cause this? Well, first of all I'd say that this analysis is a few factors short of anything scientific. For instance, the bike lane map for Edmonton likely includes lanes and paths that haven't existed for the entirety of 2009-2014, or have since been removed, so some of the injuries from my analysis are likely classified inaccurately. As well, <a href="http://ajph.aphapublications.org/doi/abs/10.2105/AJPH.2012.300762?journalCode=ajph&">other researchers</a>, when investigating bike lane safety, controlled for the presence of parked cars on the side of the road, which I did not. So while I wouldn't necessarily go so far as to say that my analysis shows Edmonton bike lanes are <i>more dangerous</i> than streets without bike lanes, I stand by the assertion that bike lanes <i>aren't safer</i> than streets without them. I embrace the subtlety of that distinction.<br />
<br />
Regardless, the data is quite clear about the effects of vehicle traffic on bike incidents, and the effects of physically separating bike paths from roads. Namely, separating vehicle and bicycle traffic may reduce bicycle injuries by a factor of 2 on busy roads, and up to a factor of 6 on quieter roads.<br />
<br />
Again this is not surprising at all - I can't stress just how intuitive and likely boring the main finding here is. But this data set of cycling injuries from 2009-2014 does seem to show that painted bike lanes have not had the effect that was perhaps intended.<br />
<br />
In my opinion, having decent bicycle infrastructure is absolutely important to having a vibrant and healthy city. Hopefully future bike lane decisions are made keeping injury prevention and statistics in mind, in such a way that we can expand our biking infrastructure as effectively as possible.Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com9tag:blogger.com,1999:blog-3055604501732306258.post-14970577376760503122016-07-03T20:10:00.002-06:002016-07-04T08:47:08.927-06:00Edmonton City Council Votes (Part 2)A year ago I did a short piece looking at Edmonton <a href="http://www.mikerobe007.ca/2015/06/city-council-analysis.html">city council voting patterns</a>. It was pretty fun and showed some cool blocks in city council, but since then we've had a <a href="http://www.mikerobe007.ca/2016/02/edmonton-ward-12-by-election-results.html">monster by-election</a>, so it seemed like now is a good time to take a second look at this analysis.<br />
<br />
Since council as a whole got elected in 2013, there have been 5763 votes performed, according to the city's <a href="https://data.edmonton.ca/City-Administration/2013-2017-Council-And-Committee-Meetings-Voting-Re/fwq5-ux79">Open Data catalogue</a>. Of course, many of these are procedural matters, and the vast majority of them are unanimous. If we restrict the votes to non-unanimous votes to see how the councillors interact, we're actually only left with 358 votes to look at.<br />
<br />
Of those 358 votes, we can come up with this result, showing how often each member of council agreed with each other member of council. I've colour-coded it to make the numbers seem a little less daunting:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGjE5ZeFYsdI3koFEcEOaaP5eDIzJlI9skMaSP1s4-OVYO_qpp9Qa9S71d8P41yXV82mAQNfzj2doDCfVz_ho1Nrd8FfDR_oaPP60hkKZbHOR2RfaNfz9ntynqbwR4yZvFGlHFcptmRkY/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="107" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGjE5ZeFYsdI3koFEcEOaaP5eDIzJlI9skMaSP1s4-OVYO_qpp9Qa9S71d8P41yXV82mAQNfzj2doDCfVz_ho1Nrd8FfDR_oaPP60hkKZbHOR2RfaNfz9ntynqbwR4yZvFGlHFcptmRkY/s400/Capture.PNG" width="400" /></a></div>
<br />
The major update here, of course, is the addition of Councillor Banga to the mix. He seems to generally follow the Iveson/Esslinger/Walters group that we identified last year, though generally less so than his predecessor Amarjeet Sohi did. He also seems to disagree with Councillor Caterina disproportionately relative to anyone else. Again, much like a year ago, Councillor Nickel is a bit of an outsider, who agrees with his colleagues far less than anyone else does.<br />
<br />
Another way of looking at this is to make network graphs. This first one shows all connections with councillors that agree with each other at least 67% of the time (this number was chosen so that Councillor Nickel isn't left out). Feel free to play with it, it's rather fun!<br />
<div>
<br /></div>
<iframe frameborder="no" height="400" scrolling="no" src="https://www.google.com/fusiontables/embedviz?containerId=googft-gviz-canvas&viz=GVIZ&t=GRAPH&gc=false&gd=false&sdb=1&rmax=100000&q=select+col0%2C+col1%2C+col2+from+1beUbgsKoV_S0flEZxv_X0vSSR-kFPPLT7loWWSj6+where+col2+%3E%3D+0.6677+and+col2+%3C%3D+1&qrs=+and+col0+%3E%3D+&qre=+and+col0+%3C%3D+&qe=&uiversion=2&state=%7B%22ps%22%3A%221_8_d_7_b_-g_-v_a_c_-n_4_-6_x_5_-h_4_6_1b_-h_3_19_b_2_w_y_9_-1a_-d_c_-18_h_7_p_-1b_0_27_l_1_2z_t_%22%2C%22cx%22%3A48.58827409358574%2C%22cy%22%3A-20.7998732397922%2C%22sw%22%3A634.0337087825492%2C%22sh%22%3A292.50310025092705%2C%22z%22%3A1.250958887189097%7D&gco_forceIFrame=true&gco_hasLabelsColumn=true&width=525&height=400" width="525"></iframe><br />
<br />
Alternatively, we can generate a network graph based on who agrees with who the most frequently. Orange arrows (when you hover over them) indicate the most frequent agreements for each councillor, blue arrows indicate that another councillor most frequently agrees with the first, but that it isn't reciprocated.<br />
<br />
<iframe frameborder="no" height="400" scrolling="no" src="https://www.google.com/fusiontables/embedviz?containerId=googft-gviz-canvas&viz=GVIZ&t=GRAPH&gc=false&gd=true&sdb=1&rmax=100000&q=select+col0%2C+col1%2C+col2+from+1_66KCKaZsMkfnYbWyjWrkG_VroF7i6NaVrad8zla&qrs=+where+col0+%3E%3D+&qre=+and+col0+%3C%3D+&qe=&uiversion=2&state=%7B%22ps%22%3A%221_0_-3f_5_8_-2d_-i_6_4f_-m_1_-49_-r_2_45_8_3_3u_11_4_-4b_k_c_-3k_13_5_-2p_s_9_-1o_-w_7_4n_-1b_b_-4s_-1a_a_3k_1n_%22%2C%22cx%22%3A56.61390137122475%2C%22cy%22%3A-52.55156109651376%2C%22sw%22%3A583.5134563924407%2C%22sh%22%3A269.1962472478888%2C%22z%22%3A0.2510350349576412%7D&gco_forceIFrame=true&gco_hasLabelsColumn=true&width=550&height=400" width="550"></iframe><br />
<br />
This shows a bit more clearly how potential groupings look at city council. Five councillors agree with Mayor Iveson more than anyone else, and two other councillors most frequently agree with two of those five. On the other hand, the remaining 5 other councillors tend to spread out from Councillor Caterina.<br />
<br />
Of course, these two groups aren't all that different - Councillor Caterina and Mayor Iveson still vote the same on 75% of <i>contested</i> motions, so realistically they agree 98% of the time on <i>all</i> motions, but the above network graph is a nice way to dramatize it!<br />
<br />
Finally, we can also take a look at how often each member of council ends up getting the result they voted for on each motion. Again, only looking at non-unanimous votes, we have:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEisJJvMZnzSvgByKEcgocaRf329S0U34S3PV1X8unMIqQ1sRc4NaDIaFmHixryzTR3oHahqHxInF_tbgCRsLGU4Nha0BVRcoNyFJea4drG2wnZOvZ7ucEVcOM6lKZHdJxqzkXRl_v2h1OY/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEisJJvMZnzSvgByKEcgocaRf329S0U34S3PV1X8unMIqQ1sRc4NaDIaFmHixryzTR3oHahqHxInF_tbgCRsLGU4Nha0BVRcoNyFJea4drG2wnZOvZ7ucEVcOM6lKZHdJxqzkXRl_v2h1OY/s1600/Capture.PNG" /></a></div>
<br />
Impressively, Mayor Iveson ends up on the winning side of a council vote 95% of the time. In fact, of all 5763 votes performed since 2013, Mayor Iveson has only been disappointed <i>17 times</i>. There are certainly many conclusions that can be drawn from that, but at the very least nobody can say that Don Iveson has difficulties instituting the agenda he wants on council.<br />
<br />
So there you go. I plan to do another analysis like this before the next election, so stay tuned for that one!Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com10tag:blogger.com,1999:blog-3055604501732306258.post-2544204083893182402016-04-26T11:27:00.000-06:002016-04-26T11:27:10.538-06:00Which Edmonton City Councillor are You?Since the Ward 12 by-election just a few months ago, Edmonton city council has gotten into quite a few rather contentious votes. Most recently the <a href="http://www.cbc.ca/news/canada/edmonton/city-council-approves-controversial-highrise-just-off-whyte-avenue-1.3545448">Mezzo Building</a> decision left quite a few observers rather upset, but earlier council decided to scrap the proposed <a href="http://www.metronews.ca/news/edmonton/2016/02/22/edmonton-heritage-festival-wants-pricey-splash-park-cut-.html">Hawrelak Park Water Play Feature</a> (worst name ever, by the way) after being faced with price increases, and has had to face some struggles with the proposed <a href="http://www.theglobeandmail.com/real-estate/calgary-and-edmonton/green-development-of-old-edmonton-airport-site-struggling-to-take-off/article29618132/">green development in the Blatchford area</a>.<br />
<br />
With that all being said, since Councillor Banga has taken on the role, Edmonton's open data suggests that there have been <a href="https://data.edmonton.ca/City-Administration/2013-2017-Council-And-Committee-Meetings-Motions/smk2-dtnx">25 votes of council</a> that have been non-unanimous, which it turns out is more than enough that no two councillors have voted the same way on everything over the last two months (even though Councillor Oshry and Mayor Iveson gave it their best shot at 24/25). That means that, with only a few questions, we can generate a choose-your-own-adventure game in the style of a Buzzfeed quiz to see which councillor you agree the most with over the last term!<br />
<br />
Which Edmonton City councillor are you? The answer will surprise you!<br />
<br />
<div class="quizz-container" data-height="auto" data-quiz="212333" data-width="100%">
</div>
<script async="" src="//dcc4iyjchzom0.cloudfront.net/widget/loader.js"></script>
Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com9tag:blogger.com,1999:blog-3055604501732306258.post-22681244632454869322016-04-22T09:49:00.002-06:002016-04-22T09:49:18.361-06:00Edmonton Zone MapEarlier this week, there was a bit of a kerfuffle raised at City Council when they contentiously passed a motion to allow a <a href="http://edmontonjournal.com/business/commercial-real-estate/divided-edmonton-city-council-approves-16-storey-mezzo-tower-near-whyte-avenue">new 16-storey building near Whyte Avenue</a>. In order to allow the new building, they had to change some of the zoning around the area.<br />
<br />
I was curious about exactly what the distribution of zones in Edmonton look like, so I decided to see if I could find a map. Oddly enough, despite the data being available on the city's <a href="https://data.edmonton.ca/Thematic-Features/City-of-Edmonton-Zoning-Bylaw-Map/b4f2-gf2b">OpenData portal</a>, there wasn't a readily-available one to be found via Google.<br />
<br />
And maybe there's a good reason - it turns out there are over 85 different zone descriptors that the city uses, and many of the individually set zones are actually rather tiny (small parks count as their own zone, for instance). If you coloured a map based on all the different types of zones, it would be a scary kaleidoscope that wouldn't be terribly useful.<br />
<br />
So instead, I've reverted to the tried and true Sim City method and labelled things broadly as either <span style="color: #38761d;">Residential</span>, <span style="color: #351c75;">Commercial</span>, or <span style="color: #bf9000;">Industrial</span> zones. Take a look:<br />
<br />
<iframe frameborder="no" height="675" scrolling="no" src="https://www.google.com/fusiontables/embedviz?q=select+col6%3E%3E1+from+1ZcXFj1sGmxC5oMwylWBZ-NnpFAdSlBgvz7WShb3x&viz=MAP&h=false&lat=53.530381212939446&lng=-113.45741895202632&t=1&z=10&l=col6%3E%3E1&y=3&tmplt=5&hml=KML" width="525"></iframe><br />
<br />
If you've lived in Edmonton for more than a couple minutes, I'm sure that this map isn't surprising to you at all. I find it still cool to actually see things laid out like this though - it really shows you the industrial moat that surrounds Mill Woods, for instance, and specifically locates all of the strip malls we seem so fond of. (If your favorite strip mall isn't coded blue, it's most likely because many areas tend to end up as 'Site Specific Development Control Provision', which is essentially bylaw code for 'none of the above'. I didn't end up colour coding them all because there 650 of them, mostly all for different reasons...)<br />
<br />
One final thought: I'm not so sure I like the sounds of the Anthony Henday being an agricultural zone. Hopefully they keep the agriculture and the four lanes of speeding gas guzzlers a little separated...Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com12tag:blogger.com,1999:blog-3055604501732306258.post-58524755305532776582016-03-08T11:51:00.004-07:002016-03-08T16:02:25.792-07:00Gender Equality in APEGAPop quiz: Given that recent data suggests that women earn <a href="http://www.cbc.ca/news/business/wage-gap-oxfam-1.3478938">72% of what men do</a> for similar work in Canada, what is the wage gap for women in engineering in Alberta?<br />
<br />
<ul>
<li>A) 13%</li>
<li>B) 0.15%</li>
<li>C) The math depends on your agenda</li>
</ul>
<br />
You're right! The math behind the wage gap depends on what you're looking to achieve in your analysis. Congratulations!<br />
<br />
Let me explain. For the last couple of years, APEGA has published a detailed salary survey of its members. (This year, APEGA instead published an 8-page summary of the survey, and asked <a href="https://www.apega.ca/about-apega/publications/salary-survey/">$1,900</a> to share the full information with you, while also withholding the complete data from previous years. Yuck.) Fortunately, through the power of the web archive, we can access the <a href="https://web.archive.org/web/20150907041148/http://www.apega.ca/Members/Publications/salarysurvey.html">previous salary survey data</a>, which is helpfully broken down into many demographics. Let's take a look.<br />
<br />
From the 2014 salary survey, the average male engineer's salary was $125,721, and the average female engineer's salary was $109,402, for a wage gap of <b>13%</b>. Alright, we're done here. That was easy.<br />
<br />
Well, maybe not so fast. One of the biggest determinants of salary is seniority, and if seniority isn't distributed similarly between genders then that may skew the data. If we compare male to female earnings based on seniority, we get:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMuZJj_HtNptyiS0aSPXefZm3DJGNDJ84EKWaB-OuA-wgfcJ7eOUdp3fk_o2HM-KkUvm2GJMd-ShvPAHQ2n5e7-ES4gw530deEic-0l1JQhrq0izGw7_yNKqLOYKs97ph6UQKbYOdeP5U/s1600/Apega+Salary2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMuZJj_HtNptyiS0aSPXefZm3DJGNDJ84EKWaB-OuA-wgfcJ7eOUdp3fk_o2HM-KkUvm2GJMd-ShvPAHQ2n5e7-ES4gw530deEic-0l1JQhrq0izGw7_yNKqLOYKs97ph6UQKbYOdeP5U/s1600/Apega+Salary2.png" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
For salary survey purposes, an A- class would be the equivalent of a co-op student, and an F+ class would be senior management.<br />
<br />
When we look at the data like this, we see that until maybe the very top levels of senior management, male and female engineers make approximately the same salaries (within 2% one way or another). If we weight this based on the total number of engineers in each category, we actually end up with females earning 0.15% more than men on average.<br />
<br />
So on the one hand we have women earning the same as men, and on the other hand we have women making 13% less, all depending on how you look at the statistics. While things are looking good from the point of view of co-workers getting paid similarly for similar responsibilities, is there a chance that something else may be pulling back on womens' chances at the better paying jobs? We can investigate further by looking at seniority by gender:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6hfltJVNECJL17k8xQogO7Bj8bWUjxFT5_R__IT7e4FhOy8LnVeKHh8bRpnV3x7Ofz6Ajeyf_SnXdffpRhwNGWxIkyORBGeYpzJgMXhzqd21a0g6jeqbvdybdaWQyqo8FRUyca3t0q54/s1600/Apega+Salary3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6hfltJVNECJL17k8xQogO7Bj8bWUjxFT5_R__IT7e4FhOy8LnVeKHh8bRpnV3x7Ofz6Ajeyf_SnXdffpRhwNGWxIkyORBGeYpzJgMXhzqd21a0g6jeqbvdybdaWQyqo8FRUyca3t0q54/s1600/Apega+Salary3.png" /></a></div>
<br />
Alright now that's something. Women tend to average around a B to a C level, whereas men tend to average around a C to a D level. Here's a major difference, and when compared with the salary averages at each seniority level, we can see where the previously-established 13% salary difference comes from.<br />
<br />
Based on the earlier analysis, I'm pretty optimistic that <i>for the same responsibility level</i>, male and female engineers make approximately the same wages. But it's definitely worth looking into what's causing the differences in distribution of work responsibilities.<br />
<br />
Starting out, I think there are three major plausible theories. A pessimistic and sexist theory could be that men are promoted faster in the workplace, and as a result tend to sit higher in seniority (glass ceiling style). The disappointing yet potentially less sexist theory is that women, for one reason or another, leave the workplace earlier than men, and as a result there are fewer of them to take on senior management roles. And the last theory is that changes in the graduation rates of female engineers are leaving women just now catching up to men in equality.<br />
<br />
Let's examine each in turn. Each level of seniority in the APEGA salary survey also contained information on length of career post-graduation, for women and for men. If the distribution of these values doesn't line up, perhaps that tells us something. The three largest groups by seniority are B, C, and D:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1BzQh_oiwBFIbB6cHkb72sjY6OI4H-WBq_VhuVJEp2Ub1p1B-40NWEWVk0iA3cFvMnMwLt0OTBQzjnLAY56eSJvMEpRtCFB9VbFt0WVghYkR4vAQeeF2F7hyphenhyphenEDBuDu8iQKCVx0OzbHJY/s1600/Seniority+B.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1BzQh_oiwBFIbB6cHkb72sjY6OI4H-WBq_VhuVJEp2Ub1p1B-40NWEWVk0iA3cFvMnMwLt0OTBQzjnLAY56eSJvMEpRtCFB9VbFt0WVghYkR4vAQeeF2F7hyphenhyphenEDBuDu8iQKCVx0OzbHJY/s400/Seniority+B.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Well that's bang on, how about:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglzQVXYo-H22bx4NdF-ZKF4KlU04ks7IgpXtA6laq_ZmC8QwUzyZn2e8u7zEhU_kUG6C2aaf-zLxmSH2OU6g2mnhAxucyOSxATJlFp0_MmzANCKmOm2-JTz6KXYTloG_C2G2rG2eL1mqM/s1600/Seniority+C.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglzQVXYo-H22bx4NdF-ZKF4KlU04ks7IgpXtA6laq_ZmC8QwUzyZn2e8u7zEhU_kUG6C2aaf-zLxmSH2OU6g2mnhAxucyOSxATJlFp0_MmzANCKmOm2-JTz6KXYTloG_C2G2rG2eL1mqM/s400/Seniority+C.png" width="400" /></a></div>
<br />
Still pretty close. Then there's:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguPH-O64RD6QTRzX0rToMcEZ2fAiCIRryCWek5HbOnz8s1ZAIRewPNWQq22YC4IpV4LTrVwsRVlukMLpPIpl6uAY1DXOO83fE8brkndmpa9FLAkXJc5SSK74Z5BWdGku180ZbWMVxgT78/s1600/Seniority+D.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguPH-O64RD6QTRzX0rToMcEZ2fAiCIRryCWek5HbOnz8s1ZAIRewPNWQq22YC4IpV4LTrVwsRVlukMLpPIpl6uAY1DXOO83fE8brkndmpa9FLAkXJc5SSK74Z5BWdGku180ZbWMVxgT78/s400/Seniority+D.png" width="400" /></a></div>
<br />
Alright, they actually all look reasonably similar. If anything, there may be a higher percentage of younger women in D-level positions than men, similar to the higher percentage of very young men in C-level positions than women. Nothing that could quite explain a 13% wage disparity though.<br />
<br />
The second theory I suggested was that women may leave the workplace at younger ages than men, for various reasons. Here's the distribution of women at different stages in engineers' careers:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMsqTGqAWzx5HpqJB_oRlRXFJblyDKKjx0jpR1deDRK7b-e6HO_vTSKU_IIlhL8jCrsXDVqyi_YCGUNuRzTLyQPM4Jp1CEgRhFOT5heMo5j41C4H5gQSc5vMnD4PdxmSg9RcRD8iZ3dPA/s1600/APEGA+Distribution2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMsqTGqAWzx5HpqJB_oRlRXFJblyDKKjx0jpR1deDRK7b-e6HO_vTSKU_IIlhL8jCrsXDVqyi_YCGUNuRzTLyQPM4Jp1CEgRhFOT5heMo5j41C4H5gQSc5vMnD4PdxmSg9RcRD8iZ3dPA/s1600/APEGA+Distribution2.png" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgc7d7F-LkAyWr-1HjzpO0EmKLuAILzDVPaQZ_b69ZuJyVf9s6VnP0zEku2cQwef5cwoFKxQDaVZFyrNKS0Yi8GO9kJdPTiGmLJ_Rg9mcxWrQgG2YatPqbjEFJjxVcpPMRM0AePjlsQFfw/s1600/APEGA+Distribution.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgc7d7F-LkAyWr-1HjzpO0EmKLuAILzDVPaQZ_b69ZuJyVf9s6VnP0zEku2cQwef5cwoFKxQDaVZFyrNKS0Yi8GO9kJdPTiGmLJ_Rg9mcxWrQgG2YatPqbjEFJjxVcpPMRM0AePjlsQFfw/s1600/APEGA+Distribution.png" /></a></div>
<br />
<br />
Yikes. Please note though that the salary data for people who've worked 35-40 years is pretty slim, so it's not terribly unlikely that there actually are some women engineers in that demographic, and the 2014 salary survey actually over-polled the number of females which may also skew the data. Either way, we see a clear trend where older and more senior engineers are substantially less likely to be female than younger engineers.<br />
<br />
The final piece of the puzzle comes from the third theory I listed above. Graduation rates for female engineers have changed wildly over the last 40 years, as shown in this graph from the <a href="http://www.onwie.ca/about-onwie">Ontario Network of Women in Engineering</a>:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.onwie.ca/FileRetrieval/3960/EN/Percentage%20of%20Female%20Undergraduates%20in%20Engineering%20(Canada).jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://www.onwie.ca/FileRetrieval/3960/EN/Percentage%20of%20Female%20Undergraduates%20in%20Engineering%20(Canada).jpg" height="300" width="400" /></a></div>
For a wide variety of reasons, the proportion of engineers graduating in 1975 who were female was only 3.6%. This climbed to approximately 20% only in the late 1990s, where it has fluctuated a bit since then. The last graph I present to you is the comparison of graduated female engineers since graduation and the percentage of female engineers in APEGA over the same time range:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeS6NcQThfdOLmbUd74eSEJVRberA_GqeMQe11SqVqgi_wDT54UKNEthxUVJcQekgDWmScfeJba-d4aCa4HbzdTGPXQNA7tPL2uahACw0ndO2LKR7xHCV1R8QWdmKMMC_HGZ1gnnEB9YI/s1600/Last+one.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeS6NcQThfdOLmbUd74eSEJVRberA_GqeMQe11SqVqgi_wDT54UKNEthxUVJcQekgDWmScfeJba-d4aCa4HbzdTGPXQNA7tPL2uahACw0ndO2LKR7xHCV1R8QWdmKMMC_HGZ1gnnEB9YI/s1600/Last+one.png" /></a></div>
<br />
While there may be a bit of a discrepancy between Canada graduation numbers and Alberta employment numbers, I think this comparison is still valid.<br />
<br />
So what are we left with to explain the wage gap for engineers in Alberta? It appears as though a significant part of it may be due to the fact that, until relatively recently, the rate of women entering engineering education was dreadfully low. A lot of the high-paying senior management positions that are held by men simply don't have many women counterparts to be offered to, leading to an imbalance in seniority. That being said, women in engineering, certainly past the 20-years-since-graduation mark, are still lagging behind their graduation rates, suggesting that women who did graduate over 20 years ago were still more likely to leave the field than their male counterparts.<br />
<br />
Where does this leave us? Well, while things are definitely getting better, and engineering is surprisingly better than the average of other workplaces, there's always work to be done. I suspect that as the workforce ages, we'll see a narrowing of the disparity in seniority, and hopefully in the meantime we can figure out which factors lead to women leaving the field disproportionately. Only when we reach a situation where opportunities at <i>all</i> levels of engineering employment are equal will we have a truly equal environment for engineers in Alberta.<br />
<br />
Edit: It's worth noting that the APEGA salary survey does not distinguish between full time and part time, or contract or non-contract work. As a result, any potential gender disparities between these forms of employment haven't been assessed in this post, or in the APEGA salary survey as a source.Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com7tag:blogger.com,1999:blog-3055604501732306258.post-8782229855059138152016-03-01T09:47:00.001-07:002016-03-01T09:47:28.638-07:00Edmonton 2015 Federal Election ResultsAs you might remember, that big old scary 2015 Federal Election happened way back in October. However, the Government only released the <a href="http://elections.ca/content.aspx?section=med&document=feb2916&dir=pre&lang=e"><i>official </i>results</a> of the election just yesterday - go take a look at them, they're quite cool!<br />
<br />
In the meantime, here's the final official map of how each poll in Edmonton voted, in a similar format to my post regarding the <a href="http://www.mikerobe007.ca/2015/04/2012-alberta-election-results-poll-by.html">previous Alberta election</a>. Each poll is coloured by which party had the most votes there (red is Liberal, blue conservative, and orange NDP), and then darker colours indicate the party had greater than 50% support.<br />
<br />
Enjoy!<br />
<br />
<iframe frameborder="no" height="500" scrolling="no" src="https://www.google.com/fusiontables/embedviz?q=select+col20%3E%3E1+from+1H2o_a2PnQ3WR0_3oD_TRbqKOdZyd0Ouj5cVsd4Oh&viz=MAP&h=false&lat=53.55878724476181&lng=-113.53025500292966&t=1&z=10&l=col20%3E%3E1&y=2&tmplt=2&hml=KML" width="500"></iframe>Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com7tag:blogger.com,1999:blog-3055604501732306258.post-59350901875068450652016-02-24T13:26:00.002-07:002016-02-24T13:26:39.180-07:00Edmonton Ward 12 By-Election ResultsEdmonton's Ward 12 by-election took place this week, and it had a historic 32 candidates running to sit on council. There were a couple of fun outcomes from this election that are worth taking a quick look at:<br />
<br />
<b>Winning Vote Total</b><br />
<b><br /></b>
I didn't bother doing a full write-up on this, but I was curious at the beginning of the race about just how many votes it would require to win this election. With 32 candidates, technically the winner could have won with 3.125% +1 votes, which is terribly low, but I had a hunch that there would be a few front runners, and lots of trailing candidates. I decided to take a look at historical civic elections in Edmonton and Calgary, and posted this graph on twitter:<br />
<br />
<blockquote class="twitter-tweet" data-lang="en">
<div dir="ltr" lang="en">
It's a ridiculous extrapolation, but this election is ridiculous already. Winner of <a href="https://twitter.com/hashtag/ward12?src=hash">#ward12</a> should get ~21% of vote. <a href="https://t.co/jHsN1QHR91">pic.twitter.com/jHsN1QHR91</a></div>
— Michael Ross (@Mikerobe007) <a href="https://twitter.com/Mikerobe007/status/701844086811004928">February 22, 2016</a></blockquote>
<script async="" charset="utf-8" src="//platform.twitter.com/widgets.js"></script>
<br />
Obviously it was a long shot extrapolation, as very few past elections have had even half the number of candidates as this one.<br />
<br />
How well did I do? The winner, Moe Banga, got <a href="http://www.edmonton.ca/city_government/by-election.aspx">17.76% of the vote</a>, which adjusts our graph to this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYTTtH103OnWIK1H4M7fOPZmy6eDBDuaQB4nQrGvPRCbfkNEXe_NYCU382mCOJLVtuN0e0hO2WmVlShp2uraXJtNfCJX5z8weGmpzfBX1VOOLU0kdzrOcws4VZ5cRERTwmmooZhX5S8K0/s1600/Vote+Share.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYTTtH103OnWIK1H4M7fOPZmy6eDBDuaQB4nQrGvPRCbfkNEXe_NYCU382mCOJLVtuN0e0hO2WmVlShp2uraXJtNfCJX5z8weGmpzfBX1VOOLU0kdzrOcws4VZ5cRERTwmmooZhX5S8K0/s1600/Vote+Share.png" /></a></div>
<br />
I'd say that looks pretty decent. Sure, it was a long shot estimate, but it was certainly a lot less dire than the worst-case guess would have been.<br />
<br />
<b>Vote Distribution</b><br />
<br />
Sadly, because of its by-election status, there were only seven polling stations in this election, which means that mapping voting data is a bit silly. Here are the results of the <a href="http://www.edmonton.ca/city_government/documents/elections/Voting%20Station%20by%20Voting%20Station%20Unofficial%20Results.pdf">winner in each poll</a> anyway, though.<br />
<br />
<iframe frameborder="no" height="500" scrolling="no" src="https://www.google.com/fusiontables/embedviz?q=select+col1+from+1h-ArBmemBThq_0jC6yIdzw0v_eXPaGHsyA0QriyM&viz=MAP&h=false&lat=53.4413784393314&lng=-113.42167675495148&t=1&z=12&l=col1&y=2&tmplt=3&hml=GEOCODABLE" width="500"></iframe>
<br />
Again, not much to write home about. This map is a lot more boring than it could have been largely because all the vote totals tend to be quite close together, and each poll represents several neighborhoods. 7 data points just isn't enough to have fun with.<br />
<br />
That being said, we can see that Moe Banga had fairly wide-ranging support, which will be encouraging for him going into council. Irfan Chaudhry, who came in fourth, had a narrow lead in Charlesworth, Walker, and Ellerslie neighborhoods, and Laura Thibert, who came in second, had a decent lead in Larkspur and Wild Rose.<br />
<br />
<b>Election Strategies</b><br />
<b><br /></b>
Finally, when we look at the <a href="http://www.edmonton.ca/city_government/documents/elections/Voting%20Station%20by%20Voting%20Station%20Unofficial%20Results.pdf">total vote breakdown</a> a little more closely, we can get a bit of an idea as to how some of the campaigns prioritized.<br />
<br />
Take advance voting, for example. Advanced votes accounted for 27.3% of all votes cast, but were prioritized by some candidates more than others. Moe Banga, the winner, was clearly quite organized and got 37% of his vote in advance (these advance votes of his were enough to beat the total votes of 26 other candidates even). Other high-ranking candidates, like Balraj Manhas, Arundeep Sandhu, Yash Sharma, and Rakesh Patel all got over 35% of their vote in a similar way. Candidate Sam Jhajj was a clear outlier though - getting <b><u>70.3%</u></b> of his votes from advance voting.<br />
<br />
And finally, <a href="http://www.edmonton.ca/city_government/municipal_elections/special-ballots.aspx">special ballots</a> only accounted for 1.1% of all ballots, and are reserved for people who can't vote in either advanced or normal ballots. Strangely enough, three candidates (Moe Banga, Rakesh Patel, and Balrash Manhas) combined had more than half of these votes. Kudos for grabbing the out-of-towners, I suppose!<br />
<br />
That's it! And we won't have another election in Edmonton for 18 months! See you after the next one.Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com76tag:blogger.com,1999:blog-3055604501732306258.post-70199655708535059392016-01-26T16:46:00.002-07:002016-01-27T10:15:46.477-07:00Taxi Stats for EdmontonThe fight in Edmonton for Uber vs Taxis has finally reached its head today, with the final vote for the updated <a href="http://www.630ched.com/2016/01/21/city-release/">Vehicle for Hire bylaw</a> to be debated by the end of the day (barring any disruptions to council).<br />
<br />
Potentially complicating matters is that this takes place during the very beginning of the Ward 12 by-election, meaning one seat on council is vacant. This is perhaps extra complicating matters as some of the candidates for the by-election are either directly involved with the <a href="http://manhas.ca/">United Cabbies Association</a> of Edmonton or calling for a <a href="https://twitter.com/arundeepyeg/status/692083565601751040">postponement</a> of the debate, under the understanding that Ward 12 contains a <a href="http://votenav.ca/on-taxis-uber-and-the-democratic-process-a-call-for-city-council-to-delay-bylaw-17400/">disproportionate number of taxi drivers</a> relative to other wards of the city.<br />
<br />
Of course, taxi drivers are only one half of the equation, and taxi users are also important to consider. While the stat on the previous link that 35% of cabbies live in Ward 12 is only sourced to the Nav Kaur campaign, there is plenty of other information from the cab users from a <a href="https://data.edmonton.ca/Surveys/Taxi-Usage-Edmonton-Insight-Community/9unw-vz23">2014 city survey</a> that reveals some data from the consumer side of things. This first map, for instance, indicates the percentage of people who regularly take taxis in each ward*:<br />
<br />
<iframe frameborder="no" height="500" scrolling="no" src="https://www.google.com/fusiontables/embedviz?q=select+col6%3E%3E1+from+1OwzV_Ly09C0TjFnH1NBmKInh3ULBHoU1a67lD2NR&viz=MAP&h=false&lat=53.54716820372605&lng=-113.48231895239263&t=1&z=10&l=col6%3E%3E1&y=2&tmplt=2&hml=KML" width="500"></iframe>
<br />
<span style="font-size: xx-small;">Green: higher taxi usage; Red: lower taxi usage<br />*Postal codes T5C, T5S, and T6P had low survey turnout and probably should be disregarded in this map.</span><br />
<br />
As may be expected, taxis are more commonly used in the interior of the city, and less commonly used to the west and east. Councillors who might be more concerned than average about their constituents' access to taxis could include McKeen (ward 6), Henderson, (ward 8), Walters (ward 10) and Nickel (ward 11).<br />
<br />
The survey also looked at the perceived importance and satisfaction for taxis in Edmonton. Both questions were rated from 1-5, with five being the most positive (extremely important and very satisfied, respectively). The averages for each ward are:<br />
<br />
Importance:<br />
<br />
<iframe frameborder="no" height="500" scrolling="no" src="https://www.google.com/fusiontables/embedviz?q=select+col6%3E%3E1+from+1gPk_GUnp0qtLTBa_ZNz4C1PhzzKX70nV5ei6Y3Bs&viz=MAP&h=false&lat=53.485110129057276&lng=-113.45536811621099&t=1&z=10&l=col6%3E%3E1&y=2&tmplt=2&hml=KML" width="500"></iframe>
<br />
<span style="font-size: xx-small;">Green: High importance; Red: Low importance</span><br />
<br />
Satisfaction:<br />
<br />
<iframe width="500" height="500" scrolling="no" frameborder="no" src="https://www.google.com/fusiontables/embedviz?q=select+col6%3E%3E1+from+1SdqKBlFZeZrPr5eWxkFGpRdwjx822IbHINOnBli7&viz=MAP&h=false&lat=53.57041655342182&lng=-113.48626716406255&t=1&z=10&l=col6%3E%3E1&y=2&tmplt=2&hml=KML"></iframe>
<span style="font-size: xx-small;">Green: High satisfaction; Red: low satisfaction</span><br />
<br />
On average, Edmonton citizens tend to view taxi services as somewhere between moderately and very important (3.86/5), and are somewhere between somewhat dissatisfied and neutral about their experiences (2.79/5).<br />
<br />
This post is mostly not to provide opinions, but to share some of the data that the city has on taxi users in Edmonton. There are parts of the city where people regularly take taxis and think they are important, but also aren't satisfied with the service they receive, and regardless of the outcome of today's vote hopefully opening up the discussion around taxi alternatives results in a better user experience overall.Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com23tag:blogger.com,1999:blog-3055604501732306258.post-53051658236822044042015-10-09T07:59:00.000-06:002015-10-09T10:48:32.778-06:00Electoral ReformIn less than two weeks we're going to head to the polls and elect the 42nd Government of Canada. The elections process we use is pretty old and potentially outdated, and it's time that we started to take a proper look at how it works.<br />
<br />
<blockquote class="tr_bq">
A quick overview of what we currently have: on October 19th, you and the ~100,000 people who live closest to you will have the chance to pick who represents you at the House of Commons. Whoever gets the most votes between you and your neighbors wins, and becomes a Member of Parliament.</blockquote>
<blockquote class="tr_bq">
Now, most candidates in each riding are associated with a political party. When they all get to the House of Commons, they tend to stick together with other people of their party, and most of the time vote the same way. Once everyone has gotten to the House of Commons, the Governor General (representing our Head of State, the Queen of England), offers the leader of the largest elected party the opportunity to form a government, which is subsequently voted on by all members of the House. </blockquote>
<blockquote class="tr_bq">
Unless you happen to live in a riding with the leader of a major party, you never actually directly vote for who is going to be prime minister or how powerful of a government that leader will have. </blockquote>
<blockquote class="tr_bq">
If the largest party has more than half of the seats (a <i>majority</i>), they'll win this vote pretty easily and form government. If they don't, they can still form government by arranging deals with other parties, but probably won't stick around for too long. </blockquote>
<br />
Our system of electing people to represent us has its share of failures. For one thing, as a representational democracy, you'd hope that everyone is equally represented in the House of Commons, but this isn't necessarily the case. In fact, the riding with the fewest voters (<a href="http://www.elections.ca/content.aspx?section=res&dir=cir/list&document=index338&lang=e">Labrador</a> - population 26,728) has almost 5 times less population than the riding with the most (Brantford-Brant, population 132,448). This means that a voter in Labrador has 5 times the voting power of one in Brantford. The full distribution of voters per riding looks like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1lThCAOFhsjvWqaZ9_Iea8UzhNFnEptvhtVIkdnBlSXMOIq2W6UN9u9CgtpGr1tSblHsN5vOEK1JKJHhzjLUJeqj9nD55FBcurwt4JnClSLHE3uf8V7-ITQljwa1bVYnGgoA0Fgl1DIs/s1600/Rep+per+Riding.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1lThCAOFhsjvWqaZ9_Iea8UzhNFnEptvhtVIkdnBlSXMOIq2W6UN9u9CgtpGr1tSblHsN5vOEK1JKJHhzjLUJeqj9nD55FBcurwt4JnClSLHE3uf8V7-ITQljwa1bVYnGgoA0Fgl1DIs/s1600/Rep+per+Riding.png" /></a></div>
<br />
This is actually really badly distributed, and the problem is serious enough that there are often laws in place to prevent this sort of thing from happening. In Alberta, for instance, the <a href="http://www.qp.alberta.ca/1266.cfm?page=E03.cfm&leg_type=Acts&isbncln=9780779755110">Electoral Boundaries Commission Act</a> limits population deviation between provincial electoral divisions to 25% of the average. Provinces like Saskatchewan and New Brunswick limit deviation from the average to only 5%, but in federal elections the largest deviation is an overwhelming <b>73%</b> below average.<br />
<br />
(Before you despair, though, keep in mind that the US Senate assigns the same number of senators per state regardless of its population. The best-represented US senate voter has <i>66 times</i> the voting power of the worst-represented, and the largest population per senate seat is <i>510%</i> of the average.)<br />
<br />
So our system isn't really all that great at ensuring that everyone's vote is worth the same, but is it any good at reflecting people's voting choices?<br />
<br />
Not entirely. At a small scale, the winner of any given riding is indeed the candidate with the most total votes, but more often than not they don't win with a majority of votes. Over the last three years, 58% of seats have been won with less than 50% support, and the <a href="https://en.wikipedia.org/wiki/Richard_Nadeau">worst winner</a> became an elected Member of Parliament with only 29.1% support.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfeDHcO1HpdN4AyqWf8ULngICX5tVfygdyA3ybler8SthQ5AlBlHCCxX5TRC0gtTQT3unqAFk_15cywA5mek5UMQ7c6kLcaLaiw-ByP82-oFPQGTqE1vyfSWW1N0HLBJ3mhRFqdeFZPAs/s1600/Winner+Support.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfeDHcO1HpdN4AyqWf8ULngICX5tVfygdyA3ybler8SthQ5AlBlHCCxX5TRC0gtTQT3unqAFk_15cywA5mek5UMQ7c6kLcaLaiw-ByP82-oFPQGTqE1vyfSWW1N0HLBJ3mhRFqdeFZPAs/s1600/Winner+Support.png" /></a></div>
<br />
This in and of itself isn't inherently evil, but our system is designed to elect representatives for each riding, and it is hard to argue that someone with less than 50% support is always the most representative of the area. The worst case situation I mentioned earlier had a Bloc Quebecois member elected with 29% support, but if the elected MP were to vote on the issue of Quebec separation, should he listen to the 29% of people who supported him, or the 71% of people who voted for explicitly non-separatist candidates??<br />
<br />
Most issues that MPs vote on aren't that white-or-black, of course, but it's not unreasonable for MPs to be put in positions where their party line disagrees with the majority opinion of their constituents, and this is not an effective form of representational democracy.<br />
<br />
This leads to one of the biggest issues with First Past the Post electoral systems - they are prone to dishonest or strategic voting. In our current election, there is a <a href="http://www.votepair.ca/">push</a> <a href="http://www.strategicvoting.ca/">from</a> <a href="https://www.votetogether.ca/">multiple</a> <a href="http://voteswap.ca/Main_Page">groups</a> to coordinate voting in swing ridings, with the mindset that it's better to vote <i>against</i> a specific party than to vote <i>for</i> a party you actually care about. Any seat that is won with less than 50% support can be prone to strategic voting, especially if voters either don't feel their preferred candidate has a chance to win or if voters are particularly angry at the front-runner.<br />
<br />
When we zoom out a bit, though, this lack of majority support per riding can lead to larger effects on a reigonal scale. If a region of five ridings has an evenly-distributed population with, say, 40% support to one party and 30% support split between each of two other parties, the party with 40% could easily end up with candidates elected in a majority of ridings and take control easily (if that party has control of setting electoral boundaries, they can potentially do this even without being the most popular by <a href="http://www.mikerobe007.ca/2015/02/voting-districts-and-gerrymandering.html">gerrymandering</a>).<br />
<br />
This is the issue of (non-geographic) proportional representation, and on a national scale our elections haven't always turned out terribly proportionately. In fact, the last four elections look like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4tM29WUIbq7r5kY3Pj5awZIt6BQTXz5JtHvFmdtrCx4D_69cQmpoCgTeZVAd2J3TJffyiGEr3H7LraBEyE2bHKnn8TWuUluMtpvU6am7vRBgfDTl_a5LxzMix7zhnySt4cO-zLI8HGsI/s1600/Prop.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4tM29WUIbq7r5kY3Pj5awZIt6BQTXz5JtHvFmdtrCx4D_69cQmpoCgTeZVAd2J3TJffyiGEr3H7LraBEyE2bHKnn8TWuUluMtpvU6am7vRBgfDTl_a5LxzMix7zhnySt4cO-zLI8HGsI/s1600/Prop.png" /></a></div>
<br />
For the last four elections, the parties that contested all available seats (hence the lack of Bloc Quebecois in the graph, they're silly anyway) have actually very neatly traced a cubic relationship between party support and seat allocation, instead of the line that we might expect.<br />
<br />
<blockquote class="tr_bq">
Before I talk about proportional representation too much more, I'd like to acknowledge a key assumption about proportional representation: it is often assumed that a voter for a candidate from a party in one region supports that party in all regions. This is definitely not always the case - I personally have voted for candidates while disliking their party leadership, and have known others to vote strictly based on candidate and not on party at all. When people argue that a party with 40% voter support should get 40% of the power in the House of Commons, that is definitely not the motivation that has driven <i>all</i> of those votes to have been cast in the first place.</blockquote>
<br />
Nevertheless, there are certainly benefits to having a system where party power post-election reflects party support. Much like an individual MP being elected with less than half of the support of their riding, the above graph shows that it's likely (and has certainly happened in the past) for parties to end up with a majority government with less than a majority of voters' support.<br />
<br />
Hopefully by now I've shown that there are issues with the current first past the post system that we have in Canada. Not everyone's votes counts at the same value, and once they've been counted we're not represented to the best extent possible.<br />
<br />
There are three leading proposals out there that are being put forward for how to deal with some of the shortfalls of our current system that are worth discussing: Alternative Voting (AV), Mixed Member Proportional (MMP), and Single Transferable Vote (STV). There are, of course, hundreds of ways that we could change our system, but these three have a history of being considered in Canada, and are worth taking a closer look into.<br />
<br />
<b>Alternative Voting</b><br />
<b><br /></b>
AV (also called Instant-Runoff Vote) isn't officially being proposed by a political party right now, but is apparently a <a href="http://news.nationalpost.com/news/canada/canadian-politics/trudeau-announcing-plan-to-kill-first-past-the-post-by-the-next-election">leading possibility</a> in the Liberals' current plans to overhaul the electoral system.<br />
<br />
When voting in an <a href="https://www.youtube.com/watch?v=3Y3jE3B8HsE">AV election</a>, every voter has the choice to rank as many candidates as they want. All voters' first choices are added up, and if any candidate has over 50% of the votes they win. If nobody has enough, then the candidate with the lowest first-place votes is eliminated, and all votes for them are redistributed to the second-ranked choices. This process continues until eventually one candidate has over 50% of the vote and wins.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://upload.wikimedia.org/wikipedia/en/thumb/1/1c/IRV_counting_flowchart.1.png/300px-IRV_counting_flowchart.1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://upload.wikimedia.org/wikipedia/en/thumb/1/1c/IRV_counting_flowchart.1.png/300px-IRV_counting_flowchart.1.png" width="286" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://upload.wikimedia.org/wikipedia/en/thumb/1/1c/IRV_counting_flowchart.1.png/300px-IRV_counting_flowchart.1.png">Source</a></td></tr>
</tbody></table>
If we were to replace our current system with Alternative Vote tomorrow, the results in ~42% of our ridings would definitely not change at all, as historically about that many seats have over 50% support to one candidate anyway. Only ridings with very close first-round results would be likely to end up differently between FPTP and AV election systems - quite often the front-runner off the first-choice votes ends up winning.<br />
<br />
So why is Alternative Vote any better than First Past the Post? Mainly because it reduces the need of strategic voting. If your favored candidate is unlikely to win, you don't have to consider a vote for them wasted, because either the winner will win with over 50% support (and you couldn't have changed it anyway), or your vote will get redistributed to someone else of your choice. This is good inasmuch as it allows voters to vote with their conscience.<br />
<br />
On the other hand<a href="https://www.youtube.com/watch?v=bWGtjqv19ZA">,</a> all the rest of the problems that exist in First Past the Post still exist in Alternative Vote. An analysis of the 2015 UK election, for instance, suggests that the results would have been <a href="http://www.electoral-reform.org.uk/blog/system-crisis">even more disproportionate</a> under AV than FPTP, and while it's true that voters can't be left with a candidate winning with 30% support, if the 50% benchmark is hit through a series of second, third, or fourth-place votes then we still have a situation where the majority of voters never picked their representative, or only picked them as a compromise solution. Keeping a system of single-winner ridings with a minimum 50% support doesn't ensure proportionality at all.<br />
<br />
<b>Mixed Member Proportional</b><br />
<b><br /></b>
MMP has been proposed by the NDP as an <a href="http://www.ndp.ca/news/ndp-wants-to-implement-mixed-proportional-system">election promise</a> this year, to be implemented by the next election. It has been proposed by the <a href="http://campaign2015.fairvote.ca/wp-content/uploads/2015/07/Law-Commission-of-Canada-Report.pdf">Law Commission of Canada</a> and independent <a href="http://www.gov.pe.ca/photos/original/er_premier2003.pdf">provincial commissions</a>, but has also officially been rejected by the voters of <a href="https://en.wikipedia.org/wiki/Ontario_electoral_reform_referendum,_2007">Ontario in 2007</a>.<br />
<br />
Youtube again does a very good job of explaining it <a href="https://www.youtube.com/watch?v=QT0I-sdoSXU">here</a>, but the quick explanation is that when a voter goes to vote in an MMP system, they get two ballots. One is the exact same as our current ballot, and the other is strictly for party preference. After all votes are cast, whoever ends up with the most votes in each riding from the first ballot gets elected, just as normal.<br />
<br />
After that, the proportion of party support received off the second ballot is compared to the number of seats won from the first ballot, and additional seats are filled from pre-existing party lists until the overall proportion of seats in the House of Commons matches the proportion from that second ballot. Overall, half of the seats in the House of Commons would be filled by each ballot.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://jackwebster.com/vote/b.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://jackwebster.com/vote/b.gif" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://jackwebster.com/vote/ball_demo.htm">Source</a></td></tr>
</tbody></table>
<br />
MMP is touted as providing a good balance between keeping geographical representation, while guaranteeing partisan proportionality and avoiding the need for strategic voting. However, it misses the mark on most of these.<br />
<br />
The issues mentioned before regarding First Past the Post winners not representing a large proportion of their constituents absolutely have not disappeared in MMP - this would leave the same number of voters without a local representative in parliament as before. Strategic voting for or against candidates at a local level would still occur too - in fact, MMP is at best a half fix from First Past the Post, since half of the ballot takes place in the exact same way as always. (It may be worse than FPTP, since the directly-elected MPs would be responsible for ridings double in size from previous ones, in order to make room for the new list-only MPs.)<br />
<br />
So let's focus on the second half of the ballot. By filling seats based on a candidate's ranking on a party list, instead of electing them directly, a whole second class of MP would be created. These MPs wouldn't be accountable to any citizens or have to represent them, since they would owe the existence of their jobs solely to the popularity of their party.<br />
<br />
<i>In my opinion</i>, this is less of an issue in countries that currently use MMP, like New Zealand, Germany, and Scotland, than it is in Canada, mostly because combined those three countries cover an area only a little bigger than Alberta. If a party ends up being elected largely without direct constituency seats, then their MPs in the country capital may not be the best suited to debate issues taking place thousands of kilometers away as opposed to only hundreds of kilometers away. The lessened importance of geographical representation in MMP becomes more of an issue when your country sprawls across five and a half timezones.<br />
<br />
The second ballot of MMP also isn't immune from strategic voting. For instance, in the <a href="https://en.wikipedia.org/wiki/Albanian_parliamentary_election,_2005">2005 Albanian election</a> the two major parties convinced their supporters to vote for them on the constituency ballots, and to vote for smaller coalition partner parties on the party ballot. This resulted in an unbalanced situation where previously tiny parties shared all the party ballot seats, and the large parties shared the constituency seats, and a pre-determined coalition took control. Similarly, in the <a href="https://en.wikipedia.org/wiki/Lesotho_general_election,_2007">2007 Lesotho election</a>, the major parties split in two and used decoy parties for the list seat votes. This example of gaming the system resulted in one party taking 69% of the seats with 52% of the vote.<br />
<br />
Obviously these are in the past and the development of a new electoral system could hopefully take steps to avoid loopholes like this, but other concerns still exist. MMP ballots tend to fall into two camps - open lists or closed lists. A closed list would trust the parties to fill in their top-up seats at their discretion, creating the possibility of career politicians who keep getting elected based on value and loyalty to the party, whereas an open list (as <a href="http://craigscott.ndp.ca/if-you-wish-to-see-proportional-representation-in-canada-vote-ndp">favoured by the NDP</a>) would have voters choose the list of MPs-to-be. Needless to say, this option would take a "simple two ballot" system and add an enormous amount of ranking to it.<br />
<br />
Mixed Member Proportional voting system tries to be the best of both worlds, but doesn't necessarily excel at either. Its local representation has all the issues we already have, and its push for proportionality creates a second class of MP with questionable accountability. Both MMP and AV appear to be half solutions to the current issues with FPTP, but they seem to address separate halves of the issue.<br />
<br />
<b>Single Transferable Vote</b><br />
<b><br /></b>
STV was recommended by the <a href="http://participedia.net/sites/default/files/case-files/653_265_final_report.pdf">Citizens' Assembly of British Columbia</a> in 2004, and got 57.7% percent support in a 2005 referendum. But the BC government said they wouldn't be bound by anything lower than <a href="http://ojs.unbc.ca/index.php/cpsr/article/view/251/301">60% support</a>, campaigned hard against it, and when it came back to referendum in 2009 it only got 39.1% support.<br />
<br />
Yet again, Youtube has a <a href="https://www.youtube.com/watch?v=l8XOZJkozfI">good explanation</a>, but what happens in STV is that ridings are merged together, with several winners getting elected from each riding. Because of this, each party can nominate multiple candidates. Voters then rank as many candidates as they want in each riding.<br />
<br />
Based on the number of seats to be filled in each new riding, a <a href="https://en.wikipedia.org/wiki/Droop_quota">minimum threshold of votes</a> is needed to get elected. Then an adapted version of Alternative Vote takes place - any candidates with more than that the threshold are elected, and their surplus votes are redistributed at a <u>fractional</u> value to the next-ranked choices. This repeats until no candidate has reached the threshold, at which point the lowest-ranked candidate is eliminated, and voters' subsequent choices are redistributed at full value.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://lorrie.cranor.org/pubs/diss/img13.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://lorrie.cranor.org/pubs/diss/img13.gif" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://lorrie.cranor.org/pubs/diss/img13.gif">Source</a></td></tr>
</tbody></table>
The use of fractional vote redistribution for winners ensures that voters who pick popular candidates aren't penalized. If a majority of the population all pick candidates from Party A, then effectively the most popular candidate from the party will share their surplus votes to voters' subsequent choices, so that voters don't have to worry too badly about the order in which they rank multiple more-or-less equal choices. Similarly, candidates who are eliminated have their votes redistributed at full value so that their voters aren't penalized either.<br />
<br />
So how does this fix anything? Well first of all, the issue in FPTP where people could feel unrepresented is very nearly eliminated. Imagine a riding that elects five MPs - in this case, each MP would need 16.67% of the vote (plus one vote) to get elected. This is the minimum threshold that five MPs could each hold, but that six people couldn't (because of the +1 vote requirement). This means that, at the absolute worst case scenario, 83% of everyone's ballots would go directly or indirectly towards electing someone, and almost always the people elected would be the first or second choices of the voters.<br />
<div>
<br /></div>
<div>
Strategic voting would no longer be an issue. Unlike FPTP, if a voter's preferred candidate doesn't get elected, that isn't going to jeopardize the chances of a compromise candidate. Voters could vote with a clear conscience, and smaller parties would not have to worry about losing votes to strategic coordination.<br />
<div>
<br /></div>
<div>
Also, the more seats there are in each riding, the closer to proportional the overall result would become. If 40% of people vote Party A in a 5-person riding, then only the two most popular Party A candidates will get elected, and all the votes cast for Party A candidates won't be redistributed anymore because they've succeeded in getting seats filled. Smaller parties will have a better chance of getting elected by the popularity of individual candidates, as the threshold to get elected to a multi-seat riding will be lower.</div>
<div>
<br /></div>
<div>
While the national results will be very close to proportional, proportionality isn't the end-goal of STV like it is in MMP. That means that voters who vote based on party affiliation can lump their ranking by party, but voters who vote based on candidates would be free to do so as they wish.</div>
<div>
<br /></div>
<div>
STV keeps geographical representation and proportional representation. So why isn't it being proposed by any major parties?</div>
<div>
<br /></div>
<div>
Well for starters, those reports I mentioned before supporting MMP were a bit weakly worded. The <a href="http://www.gov.pe.ca/photos/original/er_premier2003.pdf">PEI report</a> liked both systems, but felt that MMP would be easier to swallow for Canadians since it's only changing half of the system (pg 98). The <a href="http://campaign2015.fairvote.ca/wp-content/uploads/2015/07/Law-Commission-of-Canada-Report.pdf">Law Commission</a> report considered STV for about half a page (pg 103) before rejecting it based on it not having "geographical representation ... effective/accountable government ... [or] regional balance" without explanation. To their credit, neither the Liberals nor the Green Party have dismissed STV, and instead officially are proposing to form committees to investigate the best way forward.</div>
<div>
<br /></div>
<div>
Most complaints about the STV system are that it is complicated to explain and results in long ballots. These are definitely true, though as mentioned before an open list MMP system would likely have a similarly long ballot, and the idea of ranking people is the same as in AV. STV is also certainly not the <a href="https://en.wikipedia.org/wiki/Schulze_method">most complicated voting system</a> out there, but often the complications are required to avoid strategic voting openings.</div>
</div>
<div>
<br /></div>
<div>
Canada's electoral system is old and showing some signs of wear and tear. Both the disproportionate and inadequate representation that we have in our current system can and should be fixed, and the sooner that this happens, the better. But we don't have to accept only the Alternative Vote being entertained by the Liberals or the Mixed Member Proportional system offered by the NDP - they are at best half a fix, and at worst no better than our current system. If we're going to fix this, we should do this right.</div>
Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com6tag:blogger.com,1999:blog-3055604501732306258.post-68461871485369504712015-08-07T10:57:00.001-06:002015-08-07T10:57:16.473-06:00Fortune Favours the OldI had my birthday recently and, much like <a href="http://www.mikerobe007.ca/2014/08/the-great-oven-mitt-review-of-2014.html">last year</a>, got a joke present from a friend. This year though, it came with an explicit challenge to do something statistical with it. So for this blog post, my subject matter will be based on this box of fortune cookies:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgt0YuZiRCN1Gr-iR2WuvkGqy02INmdfRzrpxlpVcucXLrJm3ldrrTD6xAaOa_V9n4u2HZnEZxA4Bo6n-loI7OeQaYOhqskUDin9N-xI4qcQGQwbAMGhfuYUbJhuCpbWBXgcO1RvAp7rF8/s1600/Photo+2015-08-04%252C+19+40+59.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgt0YuZiRCN1Gr-iR2WuvkGqy02INmdfRzrpxlpVcucXLrJm3ldrrTD6xAaOa_V9n4u2HZnEZxA4Bo6n-loI7OeQaYOhqskUDin9N-xI4qcQGQwbAMGhfuYUbJhuCpbWBXgcO1RvAp7rF8/s400/Photo+2015-08-04%252C+19+40+59.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">My willing victim</td></tr>
</tbody></table>
<br />
So what stats can be pulled out of a box of fortune cookies? First of all, I suppose the box says there are approximately 25 cookies, but in reality it came with 38 fortunes. Ridiculous quality control, let me tell you!<br />
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicNIv9uHr0DYUAVr1fm0IEWlxOvijfjEkaTY-BtNknH-aMfQoiNMZBKnRLPAG5oDxq2nT9ow2SbAXfAlVTVZn_UnfT4RK-0SlUOhbYhbqWnJH12hK_sARMW_bg_x3wYN_SKDGiuzcrqmU/s1600/Photo+2015-08-04%252C+19+41+23.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicNIv9uHr0DYUAVr1fm0IEWlxOvijfjEkaTY-BtNknH-aMfQoiNMZBKnRLPAG5oDxq2nT9ow2SbAXfAlVTVZn_UnfT4RK-0SlUOhbYhbqWnJH12hK_sARMW_bg_x3wYN_SKDGiuzcrqmU/s400/Photo+2015-08-04%252C+19+41+23.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Of course, most fortunes just floated around without cookies</td></tr>
</tbody></table>
Fortunately, each fortune has a set of numbers on the back. Numbers are good, so let's do stats with those and leave the yummy cookie bits for later.<br />
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIsYY5lpQX21rp65ettWTKGiQwdQ6ORk_6VzXeYrenX7zYiBtkUV23bWhstVeJ3GSx3dF3jKs5qBTYRTD2pgxl9SD0zSdWtdtEjrXAFwX-D9npmLsuIqlsLbqpSfOHblG-PkgNgI0gaTY/s1600/Photo+2015-08-04%252C+19+49+08.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIsYY5lpQX21rp65ettWTKGiQwdQ6ORk_6VzXeYrenX7zYiBtkUV23bWhstVeJ3GSx3dF3jKs5qBTYRTD2pgxl9SD0zSdWtdtEjrXAFwX-D9npmLsuIqlsLbqpSfOHblG-PkgNgI0gaTY/s400/Photo+2015-08-04%252C+19+49+08.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fortunes not necessarily to scale.</td></tr>
</tbody></table>
Each fortune has a series of 6 ascending, non-repeating integers on the back. Presumably these are lucky numbers for your next lottery, but given just this set of numbers we can't necessarily tell <i>which</i> lottery they might be meant for. But can we make an educated guess?<br />
<div>
<br /></div>
<div>
<b>Quick history lesson</b>: in World War II, the Allies were at least somewhat concerned with estimating how many tanks Germany was building in any given month. One way they had was conventional espionage, which suggested that the Germans were building approximately 1,400 tanks every month between June 1940 and September 1942 (a lot of tanks). Of course, spies sometimes lie (it's their job, after all), so the second way the Allies had to estimate tank production was using statistics on captured tanks.</div>
<div>
<br /></div>
<div>
Every tank had a whole bunch of parts, and every part had a serial number stamped into it during production. These serial numbers were unique for every tank, and in the case of the gearboxes in particular, fell in unbroken sequences. Based on the distribution of serial numbers, a relatively simple formula could give an estimate of the total number of tanks produced. For instance, if the Allies saw that the tanks they destroyed in a given month were tanks produced #25, #94, #141, and #198 of that month (and were confident they were destroying them randomly), they'd be much less worried than if they destroyed tanks #52, #306, #519, and #1058.</div>
<div>
<br /></div>
<div>
It actually turned out way more accurate than anyone hoped - statistical estimates for tank production between June 1940 and September 1942 were 246 tanks per month, and in reality the Germans produced 245. Yay stats!</div>
<div>
<br /></div>
<div>
So like the famous <a href="https://en.wikipedia.org/wiki/German_tank_problem">German tank problem</a>, looking at a fortune cookie's string of numbers can give us an estimate of the total number of 'lucky numbers' that the fortune cookies might offer. In the above example, there are 6 numbers decently evenly spaced between 2 and 47. A frequentist statistical approach, therefore, suggests that the total number of possible numbers that could be on the backs of these fortune cookies is 53.83, with a 95% confidence interval of 47-77. Not terribly precise when looking at a single fortune. Another fortune might have a series of numbers with a likely maximum of 48, for instance, and if we look at the average of all 38 fortunes in the box, the average 'expected number' ends up being 49.4. And in fact, of all 38 fortunes, all numbers on the backs were between 1 and 49.</div>
<div>
<br /></div>
<div>
So we have six numbers, chosen between 1-49. Sounds like we're playing <a href="https://www.wclc.com/games/lotto-649.htm">Lotto 6/49</a>!</div>
<div>
<br /></div>
<div>
Here's what the distribution of all lucky numbers ended up being:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQMEpWfUj4wjJUfJsYVsgTXVCv8w4J7USMI8YkL3kfnEaLwwHd6i75BzdorSZLzqMLmHAM_7NY5Z_TOs-6lhGNehUZrJdOSumwdvXeq9J-8xiXbjoWjeyP9mSp0aZxlzAOneqhh9moyGw/s1600/Lucky+Numbers.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQMEpWfUj4wjJUfJsYVsgTXVCv8w4J7USMI8YkL3kfnEaLwwHd6i75BzdorSZLzqMLmHAM_7NY5Z_TOs-6lhGNehUZrJdOSumwdvXeq9J-8xiXbjoWjeyP9mSp0aZxlzAOneqhh9moyGw/s1600/Lucky+Numbers.png" /></a></div>
<div>
<br /></div>
<div>
It kinda looks like number 37 comes up way more often than the rest, and numbers 9 and 13 are super under-represented. Is this a conspiracy, or random chance?</div>
<div>
<br /></div>
<div>
With 49 numbers to choose from, 6 different numbers on each fortune, and 38 fortunes to choose from, we'd expect an average of 4.65 of each number to show up. With an expected 4.65 of each number, we can create a <a href="https://en.wikipedia.org/wiki/Poisson_distribution">Poisson distribution</a> to see how often we'd expect any given number to turn up, and see if ours is indeed random. That'd give us something like this:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvizY9JWYImC3OoqteguSd75ETliy1dQl9fBJkHqvBGzDTWw0dDepxgjuuMmmJxnbHZp0TnCnMJVAiWXx6PoZISD3rwX39E_HqhMIJulloox3NwXGdJGExhCOf2D3NmgYkmpIfFRWGmhU/s1600/Poisson.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvizY9JWYImC3OoqteguSd75ETliy1dQl9fBJkHqvBGzDTWw0dDepxgjuuMmmJxnbHZp0TnCnMJVAiWXx6PoZISD3rwX39E_HqhMIJulloox3NwXGdJGExhCOf2D3NmgYkmpIfFRWGmhU/s1600/Poisson.png" /></a></div>
<div>
<br /></div>
<div>
This suggests that the distribution of lucky numbers isn't actually all that lucky, and may be pretty much what you'd expect (R2 value of 0.82, which ain't shabby). It matches particularly closely at the tails, so having a few numbers occur 10 times each isn't all that surprising really.</div>
<div>
<br /></div>
<div>
One last analysis for the fortune cookies. Fortunes tended to come in one of three categories: advice ("Counting time is not as important as making time count"), analysis ("You are deeply attached to your family and home"), and most popularly predictions ("You will soon find something lost long ago"). Is there any relation between the type of fortune on the front, and the sum of the numbers on the back?</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAKv05Ut34AMviC7OhlSwj5GYDUzfNLiejn8q9HgRvY93_eLEVWTDkm2NQzAxWm1fGnEhJa0s-5hRFueRm4r8y7jxezCHVtgZhxlYlhsoEdOq1UZuKFxCSDJZd4ElqjDPMR5EsS5sYiQk/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAKv05Ut34AMviC7OhlSwj5GYDUzfNLiejn8q9HgRvY93_eLEVWTDkm2NQzAxWm1fGnEhJa0s-5hRFueRm4r8y7jxezCHVtgZhxlYlhsoEdOq1UZuKFxCSDJZd4ElqjDPMR5EsS5sYiQk/s1600/Capture.PNG" /></a></div>
<div>
<br />
<div>
Nope, nothing statistically significant anyway. The Analysis fortunes seem to generally have higher numbers on the back, but there are too few of them and they are too varied to be conclusive.</div>
<div>
<br /></div>
<div>
So there you go! Fortune cookies tend to have Lotto 6/49 numbers on the back that are fairly well randomly distributed. Not sure if that left any of you particularly surprised, but it's fun to know nonetheless!<br />
<div>
<br />
<br /></div>
</div>
</div>
Michael Rosshttp://www.blogger.com/profile/05782837470667904577noreply@blogger.com4