Monday, October 29, 2012

I ain't afraid of no ghosts

What do the Loch Ness monster, crop circles, and spiritualism have in common? Apart from being more or less considered fringe beliefs that have been known to spark intense facebook battles, they were all actually started off as known hoaxes, but continue to thrive to this day.

The most famous Surgeon's Photo picture of the Loch Ness monster, for instance, was produced in 1934 as supposed conclusive proof of the monster, but was revealed by the photographer to be a toy submarine sixty years later. Crop circles were started by two British men in 1978 using only simple tools, who were only revealed in 1991 after their wives got suspicious as to why they were spending so much time together at night and driving so far. Fortunately, apart from a small amount of ruined crops, neither of these hoaxes has caused any real damage - in fact I'm sure the tourism industry of Northern Scotland doesn't mind the Nessie myth at all.

Like the other two examples, the entire practice of spiritualism was also popularized by a massive hoax. Sadly, it has caused real damage.

Spiritualism largely began in the 1840s when the three Fox sisters began touring the United States claiming to be able to communicate with ghosts through mysterious 'rapping' noises. They performed séances for hundreds of people at a time and started the entire practice of communicating with lost loved ones (for a fee, of course). By the time they came clean in 1888 with signed confessions explaining how they achieved their effects, it was too late - the spiritualism movement had taken off and imitators were performing across the world.

Nowadays we live in a world where a third of Americans believe in ghosts, and more than 20% of Americans believe that mediums can talk to them. And if that wasn't bad enough, the worst aspect of this is the massive industry that has sprung up based entirely around exploiting people who've lost loved ones.

Though it is perhaps impossible to ever prove that what a medium does is fake, wouldn't a good place to start be to show that anything they can do could also be done by someone who claims no such powers? You bet it is. Fortunately, that's already been done.

Two of the most common principles put forward as explanations are known as hot reading and cold reading. Hot reading is essentially straight-up cheating - the medium would know something about a deceased individual from beforehand, tells the bereaved about it, and moves on. This is often accomplished through interactions beforehand with the individuals, planting repeat customers in large audiences, or, in one famous faith-healing example, having individuals fill out information cards beforehand and having them read out to the presenter using an ear piece.

Cold reading is much more subtle, and often involves a bit of psychology and manipulation. Commonly an on-stage medium will start with exceptionally vague statements such as "I'm getting a George. Who's George?" and will toss it out to an audience. Chances are that in an audience of a couple hundred people, either someone there will be named George, or will know a dead person named George, and if they have already paid to be there and want to contact their relative, they'd think this 'spirit' is for them. If that member of the audience were to stand up and say "My dad was named George!" they have begun to supply the medium with information and have stepped into a trap.

Once the medium has a target, there are a great number of ways they can convince the victim that they're talking to a dead person. Based on simply looking at people and reading their body language, a medium could make a series of educated guesses - if someone's in their forties to sixties they've likely recently lost a parent and most certainly don't have living a grandparent, for instance. Guesses are often stated vaguely enough that they can be recovered from if they turn out to be wrong - in this video, for instance, a medium states that a man's recently-deceased mother comes across as nervous. When she gets no response from him, she recovers by saying "... which is very unusual for her" and continues on about how extroverted she was. There's no way the medium could lose that battle.

Often mediums will make statements that could apply to almost anyone but seem less general than they are - for instance "this individual had an accident involving water" or "this person often wouldn't get stuff done because they are frustrated by the idea of mediocrity and wearied by the idea of starting over." If neither of those could potentially apply to you or someone you know then you're in a very slim minority. Statements like this are known as Barnum Statements - they are often general statements true about most people, but in a certain context can seem very personal.

A willing participant in a medium reading will take a mix of hits and misses from both cold and hot reading and will often end up dropping the misses and exaggerating the hits. When they later tell their friends about the experience, the medium will come across as having amazing powers. A general statement at the beginning of "Who's George?" gets transformed into the medium knowing that your dad was named George, even if you were the one who supplied that information. This is part of a common feature of our brains known as the confirmation bias where we tend to forget evidence that counters our preconceived notions and only remember what supports the conclusions we've already formed.

How about televised showings of live readings? Witness testimony of these suggests that often hours of material will get edited down to a half-hour show in order to boost the apparent success rate of the medium. Cheating? Definitely.

A combination of confirmation bias and Barnum statements is behind lots of other creepy phenomena like psychics, horoscopes, palm-reading, and Tarot cards. Nobody minds being told that they're creative and enthusiastic, but also patient and reliable, even though those are from different Zodiac signs (so you couldn't possibly actually be both!). 

Another favorite method of talking to the dead is the use of Ouija boards. Like crop circles, they were also started one way and have taken off to ridiculous levels of popularity. In the case of Ouija boards, they were started off and patented as a toy in 1890. That doesn't stop modern day mediums from using them to contact the dead, though. Ostensibly the spirits take over people's hands and use them to spell out enigmatic messages 'from beyond'.

Sound spooky? Good thing science can explain it. The leading explanation is known as the ideomotor effect, which is a fancy term for 'subconscious movement'. A great example of this is if you were to close your eyes and really vividly imagine tying your shoes. For a large proportion of the population, your fingers will start to twitch (ask a friend to watch - your eyes are supposed to be closed here). In the case of Ouija boards, people tend to move the cursor on the board to the letters they expect to be revealed, but can be completely unaware of the fact that they were actively moving it.

So though it can't be necessarily disproved, there truly are completely rational explanations for the effects that mediums use to convince people they can talk with the dead. So for this Hallowe'en, things turn out to not be so spooky after all.

Friday, October 5, 2012

Summer Weather: Part 2

A quick update to my post from last time!

Last week I posted the summer numbers from my weather station analysis. At that time, the scores were (out of 100):
  • Weather Network: 66.92
  • Global Weather: 66.02
  • Weather Channel: 63.99
  • Environment Canada: 55.00
  • 54.25
Based on the scoring system, a station could have gotten 100 if all of their temperature predictions were within three degrees of the actual weather, and the fraction of days with rain accurately matched the POP forecasts for every POP value (in increments of 10). A station could have gotten 0 only if their POP values were wildly inaccurate.

A better benchmark, though, is how well my system would have scored someone just guessing. That would potentially better demonstrate the effectiveness of weather forecasters.

Using historical data, I was able to create a "dummy" weather station that used previous years' averages to "forecast" the weather on a month-to-month basis. For example, every day in July was predicted to have a high of 23, a low of 12, and a POP of 60%.

The score obtained using this method? 38.12. In fact, the average temperature predictions were less accurate than every forecast in my model so far (just over 50% within three degrees), and the POP predictions was only better than half of the other stations' 5- or 6-day predictions*.

That's certainly encouraging! A weather station's forecasts even six days in the future are significantly better than the best educated guess you could make given historical data. So there you go - next time you criticize the meteorologist for being inaccurate, remember that actually, they're at least twice as good as you.

*: The method I use for scoring POP forecasts is perhaps objectively fair, but not very accommodating to different weather stations' reporting methods. Stations that give increments of 10% between 0 and 30 will necessarily do better than those who don't, even though a 10% POP forecast is more-or-less useless. I'm looking into ways to be a little bit more fair with this.

Wednesday, October 3, 2012

Mathematic Party Games

Go grab a calendar. Seriously, this will be way cooler with one. Bonus points if it’s for 2012. Got one? Trust me, you’ll want one for this. Still no? Fine, take this one. Open it up and follow along.

Riddle me this: What do 4-4, 6-6, 8-8, 10-10, and 12-12 have in common? Apart from being pairs of the same even number, that is. This is where your calendar comes in. Take a look at 4-4 (that is, April 4th). It was a Wednesday in 2012. How about 6-6 (June 6th)? Also a Wednesday. The same is true for August 8th, October 10th, and December 12th. If you don’t believe me you probably didn’t open up that calendar.

It turns our that these dates (4-4, 6-6, 8-8, 10-10, and 12-12) will always fall on the same key day within a year. In 2012 they are all Wednesdays, and in 2013 they will be Thursdays. This paves the way to a really cool party trick that I like to call “pretending to memorize a calendar”, where all you need to know is the key day for a given year. Have a friend pick a date – say the date the Mayans never said the world will end (December 21) – and in seconds you can tell them the day. In this case we know December is the 12th month, so 12-12 is a Wednesday. A week later is the 19th, the 20th is a Thursday, so the 21st is a Friday. It’s that easy!

But wait, there’s more! Those were just five easy-to-remember months – any chance there’s a similar pattern for the others? It turns out there is. The ninth of the fifth month and the fifth of the ninth month (May 9 and September 5) are also Wednesdays. That’s pretty easy to remember. Also, the eleventh of the seventh month and the seventh of the eleventh month (July 11 and November 7) are also Wednesdays. For those of you who like mnemonics, all it takes to remember that is the sentence: “I work nine to five at 7-11.”

That’s nine months of the year covered. Unfortunately March doesn’t have anything quite as easy to remember for it, but I happen to know that the nerdiest day of the year (Pi day – March 14) also falls on the same day as all these other key dates.

January and February are the only variable ones, because they have that leap day between them and other months. Fortunately enough, no matter what year it is the last day of February will also always be the same day of the week as our other key days, and we can work backwards from there. The only really tricky one is January, and even it has a pretty simple rule: three years out of four (non leap years), the 3rd of January is our key day (so a Thursday in 2013), and on the fourth year out of four (a leap year) it’s the 4th of January (Wednesday in 2012).

You are now literally seconds away from knowing the day of the week of any date for a given year – all you need to know is the one key day! Now go forth and impress people!

Quick recap for key days (within every year these dates will all fall on the same day of the week, no matter what):
January: Either the third (non-leap year) or fourth (leap year)
February: Last day of the month
March: Pi day! (fourteenth)
April: Fourth (even month)
May: Ninth (Nine to five mnemonic)
June: Sixth (even month)
July: Eleventh (7-11 mnemonic)
August: Eighth (even month)
September: Fifth
October: Tenth (even month)
November: Seventh
December: Twelfth (even month)