Thursday, June 25, 2015

City Council Analysis

Back after the 2013 Edmonton municipal election, I did a quick analysis to see if I could predict who some of the new mayor Don Iveson's friends on council would be. My thought was that councillors with similar platforms to the mayor's, and who are potentially more likely to agree with him on votes, were also likely to get voter support in the same neighborhoods due to their similarity. It seemed plausible enough so I did a correlation analysis.

It's now been a decent enough time since the last election that I've decided to check if my guess was accurate or not. Let's take a look!

Edmonton's Open Data has a log of the voting record for the 2013-2017 council, and it is fairly long. All told, there were 2,757 different motions that had been voted on so far. One issue with taking a look at all of these combined is that plenty of the votes are for procedural matters in council that get passed quickly and unanimously, and they're kinda boring from an analysis point of view. Of the 2,757 motions that have been voted on, 2,611 were unanimous one way or another. So let's ignore those, and focus on the remaining 146 contentious votes.

If we compare how each councillor and the mayor voted for every contentious vote they were present for, we can see how often any given pair end up voting the same way. The end results look like this:

(Click to zoom and enhance)
One of the first things to notice is that mayor Iveson does seem to have a fair bit of support on council - 7 councillors tend to vote the same way he does more than 80% of the time, and that's enough for a majority on most votes. These same 7 councillors (Knack, Esslinger, Henderson, Walters, Sohi, McKeen, and Loken) all tend to agree very consistently with each other too (with the possible exception of councillors McKeen and Loken). I wouldn't go so far as to say that they act as a voting block, but there certainly is evidence that they get along very well professionally, to say the least.

Other interesting observations include that councillors Loken, Caterina, and Gibbons all vote together quite a bit, with councillors Caterina and Gibbons agreeing more with each other than with the mayor. Councillors Anderson and Nickel quite clearly do not see eye-to-eye with most of the rest of their council colleagues.

One way we can check out my previous analysis is to compare the frequency councillors agree with the mayor with the correlation values I had previously obtained. If we do that, we can generate a graph like this:


Back when I did the original analysis, plenty of people (including myself) were surprised at the fact that councillor Walters ranked so low on the list. It turns out they were surprised with good reason, as he is one of the most notable outliers on the graph. It looks as though the analysis was alright, but nothing to be proud of. It is perhaps better than a random guess, but not necessarily something that provides critical or accurate insight immediately following an election.

One last graph for you. Each member of council had a fellow councillor who they tended to agree with the most. If we pretend that this coincides with who influences who, we can draw a graph like this:

This shows that for seven councillors, the person they agree with the most is mayor Iveson. The remaining councillors tend to split off in a group where they agree with the Caterina/Gibbons group that I mentioned above, though the frequency with which they agree with either of those two councillors is significantly lower than how often the rest tend to agree with the mayor.

The results from this analysis could exist due to a large number of different reasons. It's possible, for example, that this is an example of mayor Iveson's abilities to gain support from his councillors, and it's equally possible that it shows his ability to listen and accommodate the views of his councillors. Either way, it is his job to be the leader of city council, and so far the data seems to suggest he's doing just that.

Monday, June 8, 2015

NHL Odds in a Best-of-7 Series

Last year, Andrew and I worked together to look at which NHL playoff game was the most critical to victory in an NHL series. He built such a lovely database of playoff series that I just couldn't pass up the opportunity to take another look at the problem.

Before looking at real-world results, though, let's take a look at what the most important game ought to be in a perfectly even scenario. It's relatively simple to take a look at how a best-of-seven playoff series will turn out, and a Markov chain for a series will look like this:


Here you can see how each team's odds of winning the series go up or down based on how each previous game has gone. For example, if a team is leading 2 games to 1, their odds of winning the series are 69%. One important thing to note is that in this case it doesn't matter how they got there - there are three ways for a given team to get to a series score of 2-1 (count the lines if you'd like!), and they all lead to the same probability of winning the series. Also, note the symmetry in the diagram, since this model assumes both teams are perfectly even.

At this point, asking which game is most important to win becomes a rather nuanced question. We may as well ignore any sudden death games as they're obviously critical, but which of the remaining games are the most important?

Turns out, perhaps unsurprisingly, that the answer Game 5, but only if the series is tied. This game takes a team from a 50% chance of winning to 75%, cutting their opponents chances in half. This game has the single biggest change in odds one way or another.

But lets face it, teams in the playoffs aren't likely to be even, and there's a well documented home-town advantage in hockey sitting at around 54.5% over the last few seasons. If we assume only a home-town advantage (but otherwise teams are even), how does that effect the playoff model?


Surprisingly, it theoretically doesn't really change the teams' chances at the outset. In fact, the effect is rather diluted by frequently changing who plays where. This is probably good news, as it suggests that seeding order in the playoffs (which depends on teams' previous performance and is somewhat under their control) matters more in playoff series than winning home advantage.

Some differences show up between this model and the previous one, though. If Team A wins the first two games in a row at home, they have a slightly lower chance of winning overall (because they had an advantage then anyway). If team B wins or ties the first two games, they have a slightly higher chance of winning overall, because it's relatively smoother sailing for them from then on. If Team A has tied the series up after game 4, they regain a slight advantage, because they have two home games against one. All in all these differences are rather minor.

But what's far more interesting than theoretical models are actual results. Let's take a look at all playoff series since 1942 (including 14 series of the 2015 playoffs so far):


Here, Team A is both seeded higher than Team B, and has the home advantage. This results in a remarkably different set of probabilities than the first two models shown.

If the question comes back to which game is the most influential, the answer once again is quite different than the previous models. The most critical non-sudden-death game for Team A is actually Game 4 - if Team A is winning then they increase their odds by 18%, but if they're losing at that point they increase their odds by 21% and regain the statistical lead. For Team B, the most influential game is Game 5, for the same reasons as in the 50/50 model previously discussed.

It's important to note that model isn't necessarily applicable to the current Blackhawks v. Lightning Stanley Cup Final. Over half of all playoff series are from the first round of the playoffs, which until recently consisted of teams that were often extremely mismatched (as the top teams would play the 8th-ranked teams, etc.). It's not unreasonable to expect that the two teams who have made it to the Stanley Cup Final are more evenly matched than the average pairing in the first round, so I wouldn't necessarily recommend following along with the chart during this series.

Don't forget to follow along with my NHL Playoff 2015 model and cheer on the Blackhawks (who I had picked to win right the outset of these playoffs!).

Monday, May 11, 2015

Reuniting the Alberta Right

After last week's Alberta election, several of Alberta's political pundits expressed frustration that the splitting of the vote on the right may have allowed for the NDP success that we saw on election night. Danielle Smith, for instance, said:


She has a bit of a point - despite all the hype of the NDP surge during the campaign, they did still manage to get a strong majority government with less than half of the popular vote, and the combined popular vote of the two 'right-of-centre' parties could easily have beaten them.

Overall, the Wildrose Party ended up with far more seats than the PCs, even though they got 53,000 fewer votes (all this sounds like a set-up for a discussion on proportional voting systems, but I'll save that for later). Though the PC dynasty is ended for now, they certainly aren't lacking in a core voter base, and I wouldn't say they're definitely out of the game just yet.

But to those who are lamenting the splitting of the right side of the political spectrum, what's the most efficient way to reunite these two parties? If the right is to take control again, would it be easier to have the PC supporters move over to the Wildrose, or vice versa?

Let's check. I looked at the results for each riding from last week's election, and checked what the results would have been for each seat if a certain percentage of PC support moved to the Wildrose, or vice versa. First of all, let's see what happens if we increase the amount of PC voters who move over to the Wildrose: 


What this is telling us is that if 23.1% of PC supporters in each riding had instead voted Wildrose, there would have been enough to completely eliminate the PC presence in the legislature. If 35.8% of PC supporters had moved to the Wildrose, it would have been enough to take seats from the NDP and result in a majority of seats. A full reunification of the right would have resulted in 59 total seats, with 26 remaining for the NDP. In both cases, the seats won by the Liberal and Alberta Party MLAs were higher than the combined PC/Wildrose vote, so they're considered immune to this reunification effort.


On the other hand, it would have taken 30.3% of Wildrose supporters flocking back to the PCs in order to result in no Wildrose MLAs elected, and a 31.4% defection rate in order for the right to take control of a majority government.

Which one of these scenarios is most likely is a more nuanced question. Because of how poorly distributed the PC vote was between ridings, it's much easier for the Wildrose to absorb all of the PC seats (23.1% of PC support is only 95,393 voters across the province, for instance) than it is for the PC to absorb the Wildrose seats. If the goal is to reunite the right and regain control of the legislature, though, it may still be easier for the PCs to try to woo Wildrose voters - 31.4% of the Wildrose support is only 113,072 voters, and would have gotten the right back in power.

Overall, this means that a swing one way or another of about 100,000 right-leaning voters could have made all the difference in stopping the NDP from getting elected. Considering that this represents less than 8% of all voters from the last election, the possibility of a resurgence of the Alberta right is certainly not out of the question. The NDP has four years in power now to make good on their promises from the last election and retain their support, otherwise they may be in a bit of trouble during the next election.

Wednesday, May 6, 2015

Math is Difficult

Math can be difficult, so it's a good thing that Elections Alberta posts its unofficial elections results in a nice, easy-to-copy-into-Excel format!

Now that the Alberta election is done, I figured I'd post a short post just showing visually where the party support bases were located. Nothing too flashy or stats-heavy this time. Hopefully more analysis will follow!

First of all, based on unofficial results, the voter turnout last night was 57.01%. Not great, but how does that look visually?


Northern Alberta seems to have suffered the most to bad turnout, with an interesting grouping of solid turnout in the center. Both Edmonton and Calgary had poor turnout in their northeast halves for some reason. Feel free to zoom and click on the map, it's actually a lot of fun (red is low turnout, and green is high).

How about the Liberal support:




The Liberals didn't even run a full slate of candidates, so it's not terribly surprising that most of the map is blank. They did well in the one riding that they actually won, though, and did respectfully in Edmonton-Centre.

The PCs:




PC vote was surprisingly consistent across most rural areas, however that meant it was mostly consistent and low. Edmonton center and north were particularly low for the PCs, but otherwise the variation across the rest of the ridings was fairly minimal.

Wildrose:




Not terribly surprisingly, Wildrose support was concentrated in the southern rural parts of the province. As official opposition in the new government, they don't have any seats in urban ridings. This is fairly concerning, and hopefully won't create any further urban/rural divides in Alberta.

Finally, the NDP winners:




The NDP did very well in the cities and northwest rural ridings, but urban ridings south of Edmonton were more of a struggle for them. Interestingly enough, there is a substantial hole in NDP support in Calgary-Elbow, suggesting strategic anti-PC voting took precedence down there. I'm sure Greg Clark is appreciative.

There you go! Once the recount is done in Calgary-Glenmore (where it is currently tied between the NDP and PCs), I'll hopefully come back with more election analysis!

Tuesday, April 28, 2015

2012 Alberta Election Results Poll by Poll

There's only one week left until the provincial election!

I figure this as good a time as any to remind everyone of the full results from 2012. I'm going to do it a little differently than most map sources have. 

As an example, Wikipedia has this map of election results for each of the 87 electoral districts in Alberta:


This map suggests to me that two-thirds of rural Alberta voted strongly PC in 2012, rural Alberta south of Red Deer voted Wildrose, and the cities were a mix of mostly PC, Liberal, and NDP voters.

You know what's more interesting than that though? Poll by poll results. Each of the 87 electoral districts represent dozens of polls, and looking at these results can lead to a more detailed view of how people voted almost down to a neighborhood-by-neighborhood level. For instance, here's the full map of Alberta:


And here's Calgary and Edmonton:



If you're a fan of interactive maps, feel free to play with this!



For each poll, the colour represents the party with the most votes, and dark colours mean that the leading party had over 50% of the vote. Blue is for PC, red is Liberal, orange is NDP, and Wildrose is green.

The results generally follow the pattern of the overall district results, though showing significantly more Wildrose rural support up north than the original map would have us believe. As a fun exercise to the reader, I encourage you to try to find the four polls that the Alberta Party won, and the one lonely poll that the communist party won (hint: Calgary-East).

Remember to go vote next Tuesday!

Tuesday, April 21, 2015

Election Tours of Alberta

As of today, we are officially halfway through the 2015 Alberta provincial election! Has it been as much fun for you as it has been for me yet?

At the same time as we pick our Alberta government, over in the United Kingdom voters there are also going through a general election. This has gotten a fair bit more attention than the Alberta election, including a fun post from FiveThirtyEight that determines the best campaign route for the leaders of various parties in that election.

It was such a fun post, in fact, that I figured I'd try to do something similar for Alberta!

As the second half of the provincial election unfolds in Alberta, the various party leaders are going to be scrambling to get to as many events as they can in what they consider to be key constituencies across the province. But what's the best route they can take through the most key constituencies, in order to minimize driving time and get the most bang for their buck?

This is a version of the well-known travelling salesman problem, which I dealt with once before for developing an Edmonton pub crawl. For my last travelling salesman problem, I only looked at a maximum of 10 locations, which allows for a time-consuming but 100% accurate solution. With 10 locations, there are 3,628,800 possible routes between each location to be examined.

I've decided to up the ante this time, and look at 20 constituencies for each of the four parties who elected MLAs in the last election. 20 locations each means a total of 2,432,902,008,176,640,000 distance combinations would need to be checked to ensure the absolute shortest route between them all. That's not a task I'm willing to participate in...

Instead, I've fiddled around with a fun trick called simulated annealing. Essentially, you start with a random travel plan, and each iteration you compare it to a proposed one that's slightly different. If the new one is better, you swap it for the old one, but if the new one is worse then you have a probability of swapping. The probability depends on the annealing 'temperature', which decreases as you iterate the procedure.

The advantage of simulated annealing is that by occasionally allowing worse solutions, you give the system the ability to work itself out of locally optimized solutions it may have found, in order to hopefully end up finding the actual best solution.

But enough math - back to politics. For each of the four parties I looked at, I tried to find the shortest route through the 20 constituencies that each party came closest to winning or losing during the 2012 election campaign. This way, leaders were hopefully going to a mix of constituencies where they barely won and barely lost, where conceivably the appearance of the party's leader over the next two weeks might make the most difference.

Let's start with the NDP:



The closest races for the NDP were, maybe not surprisingly, mostly in Edmonton. Rachel Notley's trip would start up in Edmonton-Manning, run through 13 of Edmonton's closest races, and continue on south to Lethbridge with the occasional stop in Red Deer and Calgary. A pretty easy urban whirlwind tour for her, really.

Total time: 7 hours, 57 minutes.

Next, the Liberals:



David Swann's journey isn't altogether too different than Rachel Notley's, though it's a more even split between Calgary and Edmonton, with less in between. Unlike the NDP, there weren't any races that the Liberals won by enough of a margin that desperate help wasn't needed. The quick trip out to Canmore to deal with Banff-Cochrane ought to make for some great sight-seeing!

Total time: 9 hours, 59 minutes.

The Wildrose:



Brian Jean is in for quite a different ride. Starting up in Dunvegan-Central Peace-Notley, he only barely glances at Edmonton on his way down to the juicy urban Calgary core the Wildrose stands to gain. Then it's off to Medicine Hat before winding his way back north to Fort MacMurray.

Total time: 1 day, 1 hour, 32 minutes.

Lastly, the PCs:


The PC map looks quite similar to the Wildrose, largely because the closest contests in rural Alberta were directly between the two parties. Major differences include the four stops in north Edmonton, and the changed focus on south Calgary.

Total time: 1 day, 1 hour, 57 minutes.

So there you go! If you happen to see the campaign busses on the highway and they're not going the right way, make sure to let them know. Best of luck to all in the last two weeks of the election!

Monday, April 20, 2015

Edmonton's NHL Draft Lottery Luck

This weekend, the hockey world lit up with the news that, for the fourth time in six years, Edmonton got the first overall pick in the NHL draft lottery. This year was extra special, as the projected first-round pick Connor McDavid is supposed to be the chosen one who will lead us from our years of darkness (...or something).

The question I was faced with is just how unlikely is it that Edmonton came first 4 times in the last six years. After all, it is a lottery. The fact that the team who gets the first overall draft pick is randomly determined each year is good because it hopefully reduces the chances of a team tanking on purpose to be the worst team in the league in a given year, and keeps games interesting for fans.

Over the last six years a few different odds distributions were offered for the 14 lottery teams that didn't make the playoffs. Until 2012 only the five worst teams had a chance of getting the first draft pick (the absolute worst team had a 48.2% chance), but since 2013 all 14 of the worst teams have some chance or another.

Edmonton and Carolina were the only two teams to not make it in the playoffs over all six of those years, so it stands to reason that they had the best shots at getting the first draft picks at least once or twice in that period, right? This is what happens if you actually crunch the numbers though:


It turns out Edmonton's chances of getting four first-round picks over the last six years was actually around 1.9%. This is certainly low, but not necessarily anything impossible.

There are two reasons that this may be higher than you'd think. First of all, I was looking at the chances of Edmonton winning any four of the last six drafts, not specifically the first three, losing two, and then winning the sixth. Those odds are astronomically low, but deceptive since nobody is up in arms about the lotteries Edmonton didn't win. Secondly, Edmonton's chances were so much higher than Carolina because the first three years Carolina was in the draft lottery, they weren't in a position where they could have won first pick overall (as before 2013, a team winning the lottery could only move up a maximum of 4 positions).

All told, this gives Edmonton an expected return of 1.456 overall first picks over the last 6 years, where they actually got 4. To put that in perspective, they were expected to get almost twice as many overall first picks as the next worst team over the last six years. Of the 27 teams who made at least one appearance in the draft lottery over the last six years, we have:


Realistically, this means that the luckiest teams in the draft have been Edmonton, Florida, and Colorado, and the unluckiest has probably been Columbus. The four teams at the bottom happened to have their bad seasons in years where they weren't quite bad enough to have a shot at first overall pick (poor guys).

So yes, Edmonton has gotten lucky with draft picks over the last six years, but it's not quite as impossible as it would have otherwise seemed. We were helped out by being the worst team in the league in two years where we had nearly 50% chances of winning the draft, and by generally being terrible in the rest of the years to keep our chances high. We've gotten lucky at the draft, but only by being genuinely terrible over the last six years, and I sincerely hope that trend starts to reverse soon.